[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:18 dist-test-slave-44d2 krb5kdc[32602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:18 dist-test-slave-44d2 krb5kdc[32602](info): set up 2 sockets
Aug 08 00:32:18 dist-test-slave-44d2 krb5kdc[32602](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 00:32:19.705381 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.437s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:19 dist-test-slave-44d2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:19.851862 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:40533
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:40533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:19.961798 32622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:19.962021 32622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:19.962101 32622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:19.963299 32622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:19.963339 32622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:19.963353 32622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:19.963392 32622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:19.963428 32622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:19.965161 32622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:40533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:40533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:19.965548 32622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:19.965821 32622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:19.972311 32636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:19.972393 32629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:19.973654 32633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:19.974588 32622 server_base.cc:1034] running on GCE node
I20260808 00:32:19.974915 32622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:19.975274 32622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:19.976534 32622 hybrid_clock.cc:648] HybridClock initialized: now 1786149139976454 us; error 87 us; skew 500 ppm
Aug 08 00:32:19 dist-test-slave-44d2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149139, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:19.979225 32622 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:19.979908 32622 webserver.cc:460] Webserver started at http://127.31.210.254:38113/ using document root <none> and password file <none>
I20260808 00:32:19.980235 32622 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:19.980316 32622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:19.980587 32622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:19.981828 32622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "ecc4502131444311b934c035b2c6ac64"
format_stamp: "Formatted at 2026-08-08 00:32:19 on dist-test-slave-44d2"
server_key: "7695bce884810b746782890e6071dd51"
server_key_iv: "ae607f25b08cf482c285b1c9776a78a9"
server_key_version: "encryptionkey@0"
I20260808 00:32:19.982249 32622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "ecc4502131444311b934c035b2c6ac64"
format_stamp: "Formatted at 2026-08-08 00:32:19 on dist-test-slave-44d2"
server_key: "7695bce884810b746782890e6071dd51"
server_key_iv: "ae607f25b08cf482c285b1c9776a78a9"
server_key_version: "encryptionkey@0"
I20260808 00:32:19.984292 32622 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 00:32:19.985474 32645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:19.985754 32622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:19.985869 32622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "ecc4502131444311b934c035b2c6ac64"
format_stamp: "Formatted at 2026-08-08 00:32:19 on dist-test-slave-44d2"
server_key: "7695bce884810b746782890e6071dd51"
server_key_iv: "ae607f25b08cf482c285b1c9776a78a9"
server_key_version: "encryptionkey@0"
I20260808 00:32:19.985949 32622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:20.000145 32622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:20.003307 32622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:20.003566 32622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:20.010705 32622 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:40533
I20260808 00:32:20.010794 32718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:40533 every 8 connection(s)
I20260808 00:32:20.011476 32622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:20.013090 32720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:20.016295 32720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64: Bootstrap starting.
I20260808 00:32:20.017261 32720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:20.017644 32720 log.cc:824] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:20.018592 32720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64: No bootstrap required, opened a new log
I20260808 00:32:20.018919 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 32622
I20260808 00:32:20.019078 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:20.019800 32587 external_mini_cluster.cc:1428] Setting key 5cbf96c2aeab215e4da8a3244a5bf77b
I20260808 00:32:20.020597 32720 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc4502131444311b934c035b2c6ac64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40533 } }
I20260808 00:32:20.020790 32720 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:20.020920 32720 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecc4502131444311b934c035b2c6ac64, State: Initialized, Role: FOLLOWER
I20260808 00:32:20.021147 32720 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc4502131444311b934c035b2c6ac64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40533 } }
I20260808 00:32:20.021332 32720 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:20.021401 32720 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:20.021492 32720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:20.022353 32720 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc4502131444311b934c035b2c6ac64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40533 } }
I20260808 00:32:20.022571 32720 leader_election.cc:302] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecc4502131444311b934c035b2c6ac64; no voters:
I20260808 00:32:20.022861 32720 leader_election.cc:288] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:20.022900 32725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:20.023221 32725 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [term 1 LEADER]: Becoming Leader. State: Replica: ecc4502131444311b934c035b2c6ac64, State: Running, Role: LEADER
I20260808 00:32:20.023382 32725 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc4502131444311b934c035b2c6ac64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40533 } }
I20260808 00:32:20.023455 32720 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:20.024183 32725 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecc4502131444311b934c035b2c6ac64. Latest consensus state: current_term: 1 leader_uuid: "ecc4502131444311b934c035b2c6ac64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc4502131444311b934c035b2c6ac64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40533 } } }
I20260808 00:32:20.024364 32725 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:20.024354 32727 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecc4502131444311b934c035b2c6ac64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc4502131444311b934c035b2c6ac64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40533 } } }
I20260808 00:32:20.024471 32727 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:20.024839 32739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:20.026201 32739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:20.029254 32739 catalog_manager.cc:1347] Generated new cluster ID: 92f4145f0f8c4e1389c21380d3437b72
I20260808 00:32:20.029330 32739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:20.045039 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.023568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.31.210.254:40533)
0808 00:32:20.023851 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:20.023862 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:20.023905 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:32:20.025526 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:20.025565 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:20.025693 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:20.025829 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:20.029047 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.030757 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.032757 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.033433 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.033542 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.039152 (+ 5610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:20.039158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:20.039158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:20.039182 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:20.042072 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.042866 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.042871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.042871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.043053 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.043402 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.043403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.043404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.043924 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:20.044154 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:20.044468 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 00:32:20.044716 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260808 00:32:20.045687 32739 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:20.046237 32739 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:20.056263 32739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecc4502131444311b934c035b2c6ac64: Generated new TSK 0
I20260808 00:32:20.056548 32739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:20.095640 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40533
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:20.218014 32759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:20.218293 32759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:20.218343 32759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:20.219636 32759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:20.219709 32759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:20.219755 32759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:20.221501 32759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40533
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:20.221839 32759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:20.222097 32759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:20.228540 306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:20.228621 300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:20.228550 32765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:20.228854 32759 server_base.cc:1034] running on GCE node
I20260808 00:32:20.229074 32759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:20.229343 32759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:20.230543 32759 hybrid_clock.cc:648] HybridClock initialized: now 1786149140230535 us; error 41 us; skew 500 ppm
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:20.233850 32759 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:20.234513 32759 webserver.cc:460] Webserver started at http://127.31.210.193:36207/ using document root <none> and password file <none>
I20260808 00:32:20.234769 32759 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:20.234839 32759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:20.235002 32759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:20.236232 32759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "363f6ccdd5224f3ebe565e3dd96c7c58"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "0f41de84ad5b0feba2d2cad33953ee50"
server_key_iv: "0dca91dc36ab39ffca97085f3e82b12e"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.236735 32759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "363f6ccdd5224f3ebe565e3dd96c7c58"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "0f41de84ad5b0feba2d2cad33953ee50"
server_key_iv: "0dca91dc36ab39ffca97085f3e82b12e"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.238515 32759 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 00:32:20.239646 315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.239915 32759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:32:20.240067 32759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "363f6ccdd5224f3ebe565e3dd96c7c58"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "0f41de84ad5b0feba2d2cad33953ee50"
server_key_iv: "0dca91dc36ab39ffca97085f3e82b12e"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.240146 32759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:20.256690 32759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:20.259591 32759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:20.259819 32759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:20.260336 32759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:20.260938 32759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:20.261024 32759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.261050 32759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:20.261087 32759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.276723 32759 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:43915
I20260808 00:32:20.276765 463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:43915 every 8 connection(s)
I20260808 00:32:20.277159 32759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:20.283211 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 32759
I20260808 00:32:20.283317 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:20.283491 32587 external_mini_cluster.cc:1428] Setting key 256bf4ae877125c188f8e0f91379c47a
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:20.287886 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.278230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:36779 (local address 127.31.210.254:40533)
0808 00:32:20.278300 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:20.278302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:20.278565 (+ 263us) server_negotiation.cc:407] Connection header received
0808 00:32:20.279271 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:20.279273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:20.279292 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:20.279350 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:20.280572 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.281097 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.282126 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.282336 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.282412 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.284928 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:20.284930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:20.284930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:20.284946 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:20.286388 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.286886 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.286887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.286888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.286951 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.287214 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.287215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.287215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.287334 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:20.287413 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:20.287699 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 00:32:20.287795 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:32:20.288118 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.278190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:40533 (local address 127.31.210.193:36779)
0808 00:32:20.278415 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:20.278455 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:20.279141 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:20.279439 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:20.279444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:20.280208 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:20.280421 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:20.280426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.281198 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.281199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:20.282019 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:20.282021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.282421 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.282423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:20.282549 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.283152 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:20.283165 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:20.284818 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:20.286550 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:20.286558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:20.286560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:20.286795 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:20.287025 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:20.287026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:20.287026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:20.287145 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:20.287404 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:20.287411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:20.287559 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 00:32:20.287674 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 00:32:20.287840 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":80,"threads_started":1}
I20260808 00:32:20.288878 464 heartbeater.cc:347] Connected to a master server at 127.31.210.254:40533
I20260808 00:32:20.289001 464 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:20.289222 464 heartbeater.cc:511] Master 127.31.210.254:40533 requested a full tablet report, sending...
I20260808 00:32:20.290318 32670 ts_manager.cc:194] Registered new tserver with Master: 363f6ccdd5224f3ebe565e3dd96c7c58 (127.31.210.193:43915)
I20260808 00:32:20.291513 32670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:36779
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:20.315426 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40533
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:20.418980 469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:20.419209 469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:20.419235 469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:20.420457 469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:20.420578 469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:20.420637 469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:20.422261 469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40533
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:20.422528 469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:20.422777 469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:20.428572 475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:20.428612 476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:20.428824 469 server_base.cc:1034] running on GCE node
W20260808 00:32:20.428675 478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:20.429057 469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:20.429345 469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:20.430517 469 hybrid_clock.cc:648] HybridClock initialized: now 1786149140430495 us; error 34 us; skew 500 ppm
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:20.433903 469 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:20.434594 469 webserver.cc:460] Webserver started at http://127.31.210.194:42213/ using document root <none> and password file <none>
I20260808 00:32:20.434850 469 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:20.434930 469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:20.435171 469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:20.436600 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "1f56901b0fb14070b6328cfc77c9051c"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "2bba84cab91cd2bd84fe71fb93d62bad"
server_key_iv: "2811b92df2ea1bcbba19749b4b1a2c40"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.437077 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "1f56901b0fb14070b6328cfc77c9051c"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "2bba84cab91cd2bd84fe71fb93d62bad"
server_key_iv: "2811b92df2ea1bcbba19749b4b1a2c40"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.438813 469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:32:20.439769 494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.440059 469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:20.440184 469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "1f56901b0fb14070b6328cfc77c9051c"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "2bba84cab91cd2bd84fe71fb93d62bad"
server_key_iv: "2811b92df2ea1bcbba19749b4b1a2c40"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.440268 469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:20.486627 469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:20.489418 469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:20.489668 469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:20.490156 469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:20.490582 469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:20.490648 469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.490694 469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:20.490728 469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.506278 469 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:39709
I20260808 00:32:20.506371 656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:39709 every 8 connection(s)
I20260808 00:32:20.506743 469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:20.512709 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 469
I20260808 00:32:20.512831 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:20.513021 32587 external_mini_cluster.cc:1428] Setting key 0190aee09336f897aed45bd1b9fc0187
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:20.518872 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.507712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:56217 (local address 127.31.210.254:40533)
0808 00:32:20.507764 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:20.507766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:20.508179 (+ 413us) server_negotiation.cc:407] Connection header received
0808 00:32:20.509186 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:20.509187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:20.509202 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:20.509261 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:20.510497 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.511086 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.512311 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.512644 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.512741 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.515616 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:20.515618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:20.515618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:20.515627 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:20.517284 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.517787 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.517789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.517789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.517857 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.518146 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.518147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.518148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.518274 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:20.518352 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:20.518668 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 00:32:20.518763 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:32:20.519179 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.507665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:40533 (local address 127.31.210.194:56217)
0808 00:32:20.508012 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:20.508057 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:20.509054 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:20.509362 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:20.509366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:20.510093 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:20.510370 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:20.510375 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.511211 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.511212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:20.512208 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:20.512211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.512754 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.512755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:20.513005 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.513756 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:20.513768 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:20.515510 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:20.517426 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:20.517433 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:20.517436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:20.517685 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:20.517933 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:20.517934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:20.517934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:20.518062 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:20.518355 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:20.518364 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:20.518521 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 00:32:20.518650 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 00:32:20.518845 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":134,"threads_started":1}
I20260808 00:32:20.519672 660 heartbeater.cc:347] Connected to a master server at 127.31.210.254:40533
I20260808 00:32:20.519778 660 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:20.520016 660 heartbeater.cc:511] Master 127.31.210.254:40533 requested a full tablet report, sending...
I20260808 00:32:20.520469 32670 ts_manager.cc:194] Registered new tserver with Master: 1f56901b0fb14070b6328cfc77c9051c (127.31.210.194:39709)
I20260808 00:32:20.520891 32670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:56217
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:20.547758 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40533
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:20.654481 670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:20.654686 670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:20.654711 670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:20.655871 670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:20.655915 670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:20.655972 670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:20.657486 670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40533
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:20.657727 670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:20.657965 670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:20.663754 679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:20.663801 683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:20.663885 681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:20.663952 670 server_base.cc:1034] running on GCE node
I20260808 00:32:20.664196 670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:20.664381 670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:20.665544 670 hybrid_clock.cc:648] HybridClock initialized: now 1786149140665538 us; error 35 us; skew 500 ppm
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:20.668215 670 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:20.668713 670 webserver.cc:460] Webserver started at http://127.31.210.195:44317/ using document root <none> and password file <none>
I20260808 00:32:20.668910 670 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:20.668954 670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:20.669051 670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:20.670130 670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "ef82d197acef47068a4defceb999f3a7"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "420d90781db048e33b8ddaf299feb0a9"
server_key_iv: "dc511a000f4b0117ac66e8ee0f8cb127"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.670485 670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "ef82d197acef47068a4defceb999f3a7"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "420d90781db048e33b8ddaf299feb0a9"
server_key_iv: "dc511a000f4b0117ac66e8ee0f8cb127"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.672204 670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:20.673152 693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.673363 670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:20.673514 670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "ef82d197acef47068a4defceb999f3a7"
format_stamp: "Formatted at 2026-08-08 00:32:20 on dist-test-slave-44d2"
server_key: "420d90781db048e33b8ddaf299feb0a9"
server_key_iv: "dc511a000f4b0117ac66e8ee0f8cb127"
server_key_version: "encryptionkey@0"
I20260808 00:32:20.673617 670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:20.694531 670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:20.696846 670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:20.697034 670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:20.697412 670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:20.697862 670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:20.697926 670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.697964 670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:20.697999 670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:20.711266 670 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:40955
I20260808 00:32:20.711313 845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:40955 every 8 connection(s)
I20260808 00:32:20.711740 670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:20.715029 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 670
I20260808 00:32:20.715133 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:20.715302 32587 external_mini_cluster.cc:1428] Setting key 6827ba52379a62c911a7f0d8b3d49a83
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:20.723987 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.713020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:60675 (local address 127.31.210.254:40533)
0808 00:32:20.713098 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:20.713101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:20.713420 (+ 319us) server_negotiation.cc:407] Connection header received
0808 00:32:20.714172 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:20.714174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:20.714186 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:20.714251 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:20.715594 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.716612 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.717702 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.717963 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.718027 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.720592 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:20.720593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:20.720594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:20.720602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:20.722193 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.722663 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.722664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.722665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.722767 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.723226 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.723228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.723228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.723384 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:20.723453 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:20.723735 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 00:32:20.723811 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:32:20.724025 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.713001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:40533 (local address 127.31.210.195:60675)
0808 00:32:20.713281 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:20.713321 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:20.714048 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:20.714336 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:20.714340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:20.715146 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:20.715452 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:20.715459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.716706 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.716707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:20.717614 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:20.717617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.718043 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.718044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:20.718149 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.718739 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:20.718747 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:20.720423 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:20.722344 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:20.722351 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:20.722354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:20.722581 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:20.722848 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:20.722849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:20.722849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:20.723095 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:20.723461 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:20.723468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:20.723595 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 00:32:20.723695 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 00:32:20.723824 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":128,"threads_started":1}
I20260808 00:32:20.724761 846 heartbeater.cc:347] Connected to a master server at 127.31.210.254:40533
I20260808 00:32:20.724849 846 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:20.725006 846 heartbeater.cc:511] Master 127.31.210.254:40533 requested a full tablet report, sending...
I20260808 00:32:20.725400 32670 ts_manager.cc:194] Registered new tserver with Master: ef82d197acef47068a4defceb999f3a7 (127.31.210.195:40955)
I20260808 00:32:20.725840 32670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:60675
I20260808 00:32:20.727728 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:20.755882 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.747512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.31.210.193:43915)
0808 00:32:20.747651 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:20.747659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:20.747675 (+ 16us) server_negotiation.cc:407] Connection header received
0808 00:32:20.747703 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:20.747705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:20.747719 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:20.747805 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:20.748577 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.749325 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.750157 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.750497 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.750557 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.753161 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:20.753170 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:20.753170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:20.753177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:20.754665 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.755057 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.755060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.755060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.755115 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.755348 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.755349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.755349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.755509 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:20.755621 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:20.755698 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 00:32:20.755764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
W20260808 00:32:20.756462 354 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35436
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[32602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:20.786988 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:20.779250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35448 (local address 127.31.210.193:43915)
0808 00:32:20.779337 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:20.779340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:20.779372 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:32:20.779460 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:20.779461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:20.779476 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:20.779534 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:20.780151 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.780842 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.781670 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:20.781942 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:20.782016 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:20.784404 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:20.784406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:20.784406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:20.784413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:20.785808 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.786290 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.786291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.786291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.786343 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:20.786583 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:20.786584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:20.786584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:20.786676 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:20.786767 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:20.786852 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 00:32:20.786916 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 00:32:20.787688 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 32759
I20260808 00:32:20.800892 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 469
I20260808 00:32:20.816036 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 670
I20260808 00:32:20.825034 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 32622
2026-08-08T00:32:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2591 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[867](info): set up 2 sockets
Aug 08 00:32:20 dist-test-slave-44d2 krb5kdc[867](info): commencing operation
krb5kdc: starting...
W20260808 00:32:22.076746 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:22.207650 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43147
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:43147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:22.292214 887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:22.292419 887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:22.292443 887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:22.293684 887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:22.293735 887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:22.293749 887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:22.293762 887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:22.293774 887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:22.295202 887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:43147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:22.295400 887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:22.295583 887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:22.300892 895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:22.301016 887 server_base.cc:1034] running on GCE node
W20260808 00:32:22.300853 899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:22.301152 894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:22.301416 887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:22.301724 887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:22.302976 887 hybrid_clock.cc:648] HybridClock initialized: now 1786149142302943 us; error 69 us; skew 500 ppm
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:22.309653 887 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:22.310293 887 webserver.cc:460] Webserver started at http://127.31.210.254:42751/ using document root <none> and password file <none>
I20260808 00:32:22.310595 887 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:22.310669 887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:22.310840 887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:22.311954 887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "397abf2dc1b544839c7313e78fcaa432"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "5847e5fa2767db88fb5ce0d41c71eaa1"
server_key_iv: "d2dcb05038bf23ada2a0319a3837acb0"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.312342 887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "397abf2dc1b544839c7313e78fcaa432"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "5847e5fa2767db88fb5ce0d41c71eaa1"
server_key_iv: "d2dcb05038bf23ada2a0319a3837acb0"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.313912 887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:22.314816 908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.315038 887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:22.315156 887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "397abf2dc1b544839c7313e78fcaa432"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "5847e5fa2767db88fb5ce0d41c71eaa1"
server_key_iv: "d2dcb05038bf23ada2a0319a3837acb0"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.315236 887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:22.346433 887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:22.348688 887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:22.348874 887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:22.353446 887 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:43147
I20260808 00:32:22.353497 974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:43147 every 8 connection(s)
I20260808 00:32:22.353905 887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:22.354882 977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:22.357226 977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432: Bootstrap starting.
I20260808 00:32:22.357779 977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:22.358026 977 log.cc:824] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:22.358719 977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432: No bootstrap required, opened a new log
I20260808 00:32:22.360294 977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397abf2dc1b544839c7313e78fcaa432" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43147 } }
I20260808 00:32:22.360435 977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:22.360458 977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397abf2dc1b544839c7313e78fcaa432, State: Initialized, Role: FOLLOWER
I20260808 00:32:22.360688 977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397abf2dc1b544839c7313e78fcaa432" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43147 } }
I20260808 00:32:22.360783 977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:22.360809 977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:22.360848 977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:22.361342 977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397abf2dc1b544839c7313e78fcaa432" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43147 } }
I20260808 00:32:22.361430 977 leader_election.cc:302] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 397abf2dc1b544839c7313e78fcaa432; no voters:
I20260808 00:32:22.361559 977 leader_election.cc:288] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:22.361639 981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:22.361817 981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [term 1 LEADER]: Becoming Leader. State: Replica: 397abf2dc1b544839c7313e78fcaa432, State: Running, Role: LEADER
I20260808 00:32:22.361841 977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:22.361909 981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397abf2dc1b544839c7313e78fcaa432" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43147 } }
I20260808 00:32:22.362525 982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "397abf2dc1b544839c7313e78fcaa432" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397abf2dc1b544839c7313e78fcaa432" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43147 } } }
I20260808 00:32:22.362692 983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 397abf2dc1b544839c7313e78fcaa432. Latest consensus state: current_term: 1 leader_uuid: "397abf2dc1b544839c7313e78fcaa432" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397abf2dc1b544839c7313e78fcaa432" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43147 } } }
I20260808 00:32:22.362794 983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:22.362766 982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:22.363082 995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:22.363610 995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:22.363739 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 887
I20260808 00:32:22.363837 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:22.364012 32587 external_mini_cluster.cc:1428] Setting key 726dcfd00d4df1a2d176cafe365bc08b
I20260808 00:32:22.365420 995 catalog_manager.cc:1347] Generated new cluster ID: 2404d5ae5bc2439aa0e0a654130a6e12
I20260808 00:32:22.365487 995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:22.374131 1004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.364512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.31.210.254:43147)
0808 00:32:22.364696 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.364704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.364909 (+ 205us) server_negotiation.cc:407] Connection header received
0808 00:32:22.365053 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.365061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.365120 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.365277 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:22.365975 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.366889 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.367657 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.368093 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.368235 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.370637 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:22.370641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:22.370641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:22.370654 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:22.372378 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.372839 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.372842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.372843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.373002 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.373237 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.373242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.373243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.373486 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:22.373650 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.373771 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.373924 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260808 00:32:22.377203 995 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:22.377743 995 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:22.387181 995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 397abf2dc1b544839c7313e78fcaa432: Generated new TSK 0
I20260808 00:32:22.387391 995 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:22.424654 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43147
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:22.531515 1008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:22.531749 1008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:22.531777 1008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:22.533130 1008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:22.533178 1008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:22.533208 1008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:22.534862 1008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43147
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:22.535140 1008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:22.535393 1008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:22.541642 1017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:22.541679 1008 server_base.cc:1034] running on GCE node
W20260808 00:32:22.541622 1018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:22.541649 1020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:22.542147 1008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:22.542416 1008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:22.543608 1008 hybrid_clock.cc:648] HybridClock initialized: now 1786149142543587 us; error 40 us; skew 500 ppm
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:22.546489 1008 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:22.546991 1008 webserver.cc:460] Webserver started at http://127.31.210.193:43643/ using document root <none> and password file <none>
I20260808 00:32:22.547260 1008 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:22.547338 1008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:22.547537 1008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:22.549378 1008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "80f49274865446629a45b628e9cc6cbb"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "47984974a5a0c8bfd6c4aa453921af8d"
server_key_iv: "053138d99886543cf4b5a8a50693a82c"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.549836 1008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "80f49274865446629a45b628e9cc6cbb"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "47984974a5a0c8bfd6c4aa453921af8d"
server_key_iv: "053138d99886543cf4b5a8a50693a82c"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.551450 1008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:22.552304 1036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.552551 1008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:22.552675 1008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "80f49274865446629a45b628e9cc6cbb"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "47984974a5a0c8bfd6c4aa453921af8d"
server_key_iv: "053138d99886543cf4b5a8a50693a82c"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.552768 1008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:22.571144 1008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:22.573414 1008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:22.573608 1008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:22.573889 1008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:22.574240 1008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:22.574298 1008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.574337 1008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:22.574389 1008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.581048 1008 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:36615
I20260808 00:32:22.581106 1203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:36615 every 8 connection(s)
I20260808 00:32:22.581604 1008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:22.591578 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 1008
I20260808 00:32:22.591694 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:22.591858 32587 external_mini_cluster.cc:1428] Setting key 6db2635e8f8ae295fcee806f130b85a7
I20260808 00:32:22.592021 1004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.582398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:36777 (local address 127.31.210.254:43147)
0808 00:32:22.582481 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.582483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.582740 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:32:22.583965 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.583966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.583982 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.584073 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:22.584773 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.585399 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.586460 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.586672 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.586730 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.589044 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:22.589046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:22.589046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:22.589054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:22.590513 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.590972 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.590974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.590974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.591033 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.591291 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.591292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.591293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.591411 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:22.591498 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.591821 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.591929 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 00:32:22.592201 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.582380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43147 (local address 127.31.210.193:36777)
0808 00:32:22.582590 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:22.582634 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:22.583759 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:22.584163 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:22.584167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:22.584436 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:22.584652 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.584656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.585503 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.585505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.586366 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.586369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.586755 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.586756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.586865 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.587436 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:22.587444 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:22.588937 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:22.590612 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:22.590620 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:22.590623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:22.590876 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:22.591106 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:22.591107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:22.591108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:22.591218 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:22.591491 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:22.591498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:22.591640 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 00:32:22.591789 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 00:32:22.591957 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":121,"threads_started":1}
I20260808 00:32:22.593052 1204 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43147
I20260808 00:32:22.593194 1204 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:22.593415 1204 heartbeater.cc:511] Master 127.31.210.254:43147 requested a full tablet report, sending...
I20260808 00:32:22.594056 927 ts_manager.cc:194] Registered new tserver with Master: 80f49274865446629a45b628e9cc6cbb (127.31.210.193:36615)
I20260808 00:32:22.594900 927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:36777
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:22.619728 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43147
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:22.716570 1212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:22.716846 1212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:22.716888 1212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:22.718290 1212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:22.718369 1212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:22.718430 1212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:22.720211 1212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43147
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:22.720597 1212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:22.720868 1212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:22.726817 1220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:22.726817 1222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:22.726817 1225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:22.727140 1212 server_base.cc:1034] running on GCE node
I20260808 00:32:22.727555 1212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:22.727825 1212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:22.729025 1212 hybrid_clock.cc:648] HybridClock initialized: now 1786149142729034 us; error 77 us; skew 500 ppm
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:22.732247 1212 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:22.732861 1212 webserver.cc:460] Webserver started at http://127.31.210.194:32857/ using document root <none> and password file <none>
I20260808 00:32:22.733148 1212 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:22.733202 1212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:22.733386 1212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:22.734622 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "996a222826a344ae96cf01c551de7404"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "6f363ee78267b6c522abe1fe93814a5e"
server_key_iv: "ea7ff68bc58f56ce9588ad3de8154448"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.735088 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "996a222826a344ae96cf01c551de7404"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "6f363ee78267b6c522abe1fe93814a5e"
server_key_iv: "ea7ff68bc58f56ce9588ad3de8154448"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.736905 1212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:22.737883 1235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.738147 1212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:22.738246 1212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "996a222826a344ae96cf01c551de7404"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "6f363ee78267b6c522abe1fe93814a5e"
server_key_iv: "ea7ff68bc58f56ce9588ad3de8154448"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.738310 1212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:22.770928 1212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:22.773545 1212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:22.773681 1212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:22.773913 1212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:22.774214 1212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:22.774248 1212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.774277 1212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:22.774298 1212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.780928 1212 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:38459
I20260808 00:32:22.781167 1404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:38459 every 8 connection(s)
I20260808 00:32:22.781404 1212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:22.786778 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 1212
I20260808 00:32:22.786872 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:22.787091 32587 external_mini_cluster.cc:1428] Setting key 451c14cda84d9cef0881cbd4b9ab6074
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:22.792642 1004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.782437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:39729 (local address 127.31.210.254:43147)
0808 00:32:22.782541 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.782544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.782743 (+ 199us) server_negotiation.cc:407] Connection header received
0808 00:32:22.783781 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.783783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.783796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.783849 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:22.784559 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.785083 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.786138 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.786355 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.786450 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.789072 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:22.789073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:22.789074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:22.789082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:22.790986 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.791437 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.791439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.791439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.791507 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.791816 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.791817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.791817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.791974 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:22.792047 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.792353 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.792456 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 00:32:22.792789 1407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.782377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43147 (local address 127.31.210.194:39729)
0808 00:32:22.782600 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:22.782654 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:22.783632 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:22.783925 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:22.783930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:22.784217 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:22.784423 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.784427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.785186 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.785187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.786040 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.786043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.786461 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.786463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.786609 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.787243 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:22.787254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:22.788970 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:22.791092 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:22.791100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:22.791102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:22.791352 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:22.791581 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:22.791582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:22.791582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:22.791734 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:22.792055 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:22.792061 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:22.792195 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 00:32:22.792307 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 00:32:22.792457 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
I20260808 00:32:22.793377 1405 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43147
I20260808 00:32:22.793506 1405 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:22.793711 1405 heartbeater.cc:511] Master 127.31.210.254:43147 requested a full tablet report, sending...
I20260808 00:32:22.794117 927 ts_manager.cc:194] Registered new tserver with Master: 996a222826a344ae96cf01c551de7404 (127.31.210.194:38459)
I20260808 00:32:22.794543 927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:39729
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:22.817488 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43147
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:22.903268 1411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:22.903473 1411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:22.903499 1411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:22.904677 1411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:22.904721 1411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:22.904747 1411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:22.906189 1411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43147
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:22.906378 1411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:22.906551 1411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:22.911721 1422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:22.911870 1420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:22.911913 1411 server_base.cc:1034] running on GCE node
W20260808 00:32:22.911868 1418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:22.912358 1411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:22.912654 1411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:22.913827 1411 hybrid_clock.cc:648] HybridClock initialized: now 1786149142913812 us; error 28 us; skew 500 ppm
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:22.916740 1411 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:22.917207 1411 webserver.cc:460] Webserver started at http://127.31.210.195:34293/ using document root <none> and password file <none>
I20260808 00:32:22.917452 1411 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:22.917532 1411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:22.917688 1411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:22.918859 1411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "8c034db3a5ee4382a8223213c9c4ae52"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "7aeef736760b7a1dc3a9c8ba7ec3822e"
server_key_iv: "3035b5990268ba7b011975932d8b9f3a"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.919293 1411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "8c034db3a5ee4382a8223213c9c4ae52"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "7aeef736760b7a1dc3a9c8ba7ec3822e"
server_key_iv: "3035b5990268ba7b011975932d8b9f3a"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.920898 1411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:22.921730 1434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.921911 1411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:22.922156 1411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "8c034db3a5ee4382a8223213c9c4ae52"
format_stamp: "Formatted at 2026-08-08 00:32:22 on dist-test-slave-44d2"
server_key: "7aeef736760b7a1dc3a9c8ba7ec3822e"
server_key_iv: "3035b5990268ba7b011975932d8b9f3a"
server_key_version: "encryptionkey@0"
I20260808 00:32:22.922281 1411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:22.947764 1411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:22.950316 1411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:22.950489 1411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:22.950750 1411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:22.951105 1411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:22.951165 1411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.951246 1411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:22.951275 1411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:22.958191 1411 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:35103
I20260808 00:32:22.958237 1593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:35103 every 8 connection(s)
I20260808 00:32:22.958609 1411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:22.963953 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 1411
I20260808 00:32:22.964066 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:22.964227 32587 external_mini_cluster.cc:1428] Setting key 50c4dd1c5c215037e983e29054e9a804
Aug 08 00:32:22 dist-test-slave-44d2 krb5kdc[867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:22.969266 1004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.959492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:57573 (local address 127.31.210.254:43147)
0808 00:32:22.959538 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.959540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.959795 (+ 255us) server_negotiation.cc:407] Connection header received
0808 00:32:22.960858 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.960860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.960874 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.960934 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:22.961581 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.962169 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.963319 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.963605 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.963690 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.966444 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:22.966445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:22.966446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:22.966454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:22.967766 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.968226 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.968227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.968228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.968288 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.968599 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.968600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.968600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.968723 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:22.968797 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.969096 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.969178 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:32:22.969458 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.959455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43147 (local address 127.31.210.195:57573)
0808 00:32:22.959665 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:22.959703 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:22.960735 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:22.961020 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:22.961023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:22.961285 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:22.961475 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.961480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.962279 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.962285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.963227 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.963229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.963703 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.963704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.963875 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.964556 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:22.964566 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:22.966347 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:22.967856 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:22.967863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:22.967866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:22.968143 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:22.968374 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:22.968375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:22.968375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:22.968536 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:22.968799 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:22.968807 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:22.968948 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 00:32:22.969055 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 00:32:22.969220 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":118,"threads_started":1}
I20260808 00:32:22.969942 1594 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43147
I20260808 00:32:22.970031 1594 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:22.970237 1594 heartbeater.cc:511] Master 127.31.210.254:43147 requested a full tablet report, sending...
I20260808 00:32:22.970597 927 ts_manager.cc:194] Registered new tserver with Master: 8c034db3a5ee4382a8223213c9c4ae52 (127.31.210.195:35103)
I20260808 00:32:22.970963 927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:57573
I20260808 00:32:22.976159 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:32:22.984576 1004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.978151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.31.210.254:43147)
0808 00:32:22.978189 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.978191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.978380 (+ 189us) server_negotiation.cc:407] Connection header received
0808 00:32:22.978456 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.978457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.978467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.978517 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:22.979060 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.979532 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.980266 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.980458 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.980609 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.981778 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:22.981780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:22.981780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:22.981788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:22.983259 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.983648 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.983649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.983650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.983711 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:22.983917 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:22.983918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:22.983918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:22.984111 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:22.984222 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.984360 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.984442 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:32:22.987217 927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:22.989228 927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:22.997001 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.992465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:35103 (local address 127.0.0.1:60132)
0808 00:32:22.992916 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:22.992927 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:22.992982 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:22.993283 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:22.993286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:22.993288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:22.993357 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.993360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.994694 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.994695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.995864 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.995866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.996701 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.996702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.996782 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.996796 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:32:22.996873 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:32:22.996922 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":57,"thread_start_us":97,"threads_started":1}
I20260808 00:32:22.997026 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.992562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.31.210.195:35103)
0808 00:32:22.993109 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.993113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.993121 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:22.993153 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.993154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.993166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.993257 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:22.993436 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.994575 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.995959 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.996615 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.996695 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.996748 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.996904 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.996948 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":207,"threads_started":1}
I20260808 00:32:22.997680 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.992449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:38459 (local address 127.0.0.1:34892)
0808 00:32:22.992677 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:22.992718 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:22.992790 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:22.993686 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:22.993687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:22.993689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:22.993732 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.993733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.994956 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.994957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.996219 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.996221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.997431 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.997432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.997503 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.997508 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:22.997561 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:32:22.997602 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":65,"thread_start_us":103,"threads_started":1}
I20260808 00:32:22.997871 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.992785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:36615 (local address 127.0.0.1:40228)
0808 00:32:22.993712 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:22.993722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:22.993772 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:22.993983 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:22.993983 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:22.993985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:22.994037 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.994038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.995004 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.995004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.996203 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:22.996207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.997522 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.997523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:22.997580 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.997583 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:22.997628 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:32:22.997657 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":22,"thread_start_us":82,"threads_started":1}
I20260808 00:32:22.997860 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.992926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.31.210.193:36615)
0808 00:32:22.993125 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.993129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.993839 (+ 710us) server_negotiation.cc:407] Connection header received
0808 00:32:22.993887 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.993890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.993906 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.993975 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:22.994094 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.994908 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.996557 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.997371 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.997439 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.997491 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.997721 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.997768 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260808 00:32:22.997699 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:22.992575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.31.210.194:38459)
0808 00:32:22.993443 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:22.993447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:22.993457 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:22.993500 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:22.993502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:22.993517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:22.993600 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:22.993818 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.994852 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.996308 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:22.997354 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:22.997437 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:22.997518 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:22.997573 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 00:32:22.997623 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":99,"threads_started":1}
I20260808 00:32:22.998013 1498 tablet_service.cc:1467] Processing CreateTablet for tablet 66d9dc8e049048e9b82de7f2483c6798 (DEFAULT_TABLE table=test-table [id=e6dbc3183d09437081d1b539603671b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:22.998394 1498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d9dc8e049048e9b82de7f2483c6798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:22.998534 1110 tablet_service.cc:1467] Processing CreateTablet for tablet 66d9dc8e049048e9b82de7f2483c6798 (DEFAULT_TABLE table=test-table [id=e6dbc3183d09437081d1b539603671b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:22.998880 1110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d9dc8e049048e9b82de7f2483c6798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:22.998835 1301 tablet_service.cc:1467] Processing CreateTablet for tablet 66d9dc8e049048e9b82de7f2483c6798 (DEFAULT_TABLE table=test-table [id=e6dbc3183d09437081d1b539603671b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:22.999155 1301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d9dc8e049048e9b82de7f2483c6798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:23.002012 1617 tablet_bootstrap.cc:492] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52: Bootstrap starting.
I20260808 00:32:23.002656 1618 tablet_bootstrap.cc:492] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404: Bootstrap starting.
I20260808 00:32:23.002740 1619 tablet_bootstrap.cc:492] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb: Bootstrap starting.
I20260808 00:32:23.002867 1617 tablet_bootstrap.cc:654] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:23.003438 1619 tablet_bootstrap.cc:654] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:23.003576 1618 tablet_bootstrap.cc:654] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:23.004077 1619 log.cc:824] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:23.004158 1617 log.cc:824] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:23.004200 1618 log.cc:824] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:23.004942 1619 tablet_bootstrap.cc:492] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb: No bootstrap required, opened a new log
I20260808 00:32:23.005009 1617 tablet_bootstrap.cc:492] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52: No bootstrap required, opened a new log
I20260808 00:32:23.005035 1619 ts_tablet_manager.cc:1397] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:32:23.005082 1617 ts_tablet_manager.cc:1397] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:32:23.005144 1618 tablet_bootstrap.cc:492] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404: No bootstrap required, opened a new log
I20260808 00:32:23.005214 1618 ts_tablet_manager.cc:1397] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:32:23.006508 1619 raft_consensus.cc:348] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.006767 1619 raft_consensus.cc:374] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:23.006819 1619 raft_consensus.cc:729] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f49274865446629a45b628e9cc6cbb, State: Initialized, Role: FOLLOWER
I20260808 00:32:23.006939 1619 consensus_queue.cc:260] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.007431 1619 ts_tablet_manager.cc:1428] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:32:23.007440 1204 heartbeater.cc:503] Master 127.31.210.254:43147 was elected leader, sending a full tablet report...
I20260808 00:32:23.007436 1618 raft_consensus.cc:348] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.007632 1618 raft_consensus.cc:374] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:23.007675 1618 raft_consensus.cc:729] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996a222826a344ae96cf01c551de7404, State: Initialized, Role: FOLLOWER
I20260808 00:32:23.007547 1617 raft_consensus.cc:348] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.007776 1617 raft_consensus.cc:374] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:23.007829 1617 raft_consensus.cc:729] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c034db3a5ee4382a8223213c9c4ae52, State: Initialized, Role: FOLLOWER
I20260808 00:32:23.007835 1618 consensus_queue.cc:260] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.008041 1617 consensus_queue.cc:260] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.008136 1618 ts_tablet_manager.cc:1428] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:32:23.008316 1617 ts_tablet_manager.cc:1428] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:23.008440 1594 heartbeater.cc:503] Master 127.31.210.254:43147 was elected leader, sending a full tablet report...
I20260808 00:32:23.008505 1405 heartbeater.cc:503] Master 127.31.210.254:43147 was elected leader, sending a full tablet report...
W20260808 00:32:23.032045 1406 tablet.cc:2369] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:32:23.082510 1206 tablet.cc:2369] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:23.200886 1623 raft_consensus.cc:482] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:23.201050 1623 raft_consensus.cc:504] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.201668 1623 leader_election.cc:288] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c034db3a5ee4382a8223213c9c4ae52 (127.31.210.195:35103), 996a222826a344ae96cf01c551de7404 (127.31.210.194:38459)
I20260808 00:32:23.204954 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:23.201877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:38459 (local address 127.31.210.193:49613)
0808 00:32:23.202038 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:23.202051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:23.202115 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:23.202283 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:23.202284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:23.202286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:23.202378 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:23.202380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.203077 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.203078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:23.203901 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:23.203903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.204725 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.204726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:23.204802 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:23.204808 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:32:23.204857 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:32:23.204888 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":42,"thread_start_us":65,"threads_started":1}
I20260808 00:32:23.205080 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:23.202040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:49613 (local address 127.31.210.194:38459)
0808 00:32:23.202095 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:23.202097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:23.202109 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:23.202183 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:23.202184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:23.202196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:23.202252 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:23.202466 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.202951 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.204190 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.204643 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.204720 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:23.204754 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:23.204947 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 00:32:23.204993 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:32:23.205178 1629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:23.201910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:35103 (local address 127.31.210.193:42455)
0808 00:32:23.202039 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:23.202052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:23.202117 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:23.202318 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:23.202319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:23.202321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:23.202404 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:23.202406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.203072 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.203073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:23.203898 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:23.203900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.204793 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.204799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:23.204966 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:23.204972 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:32:23.205043 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:32:23.205081 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":19,"thread_start_us":64,"threads_started":1}
I20260808 00:32:23.205192 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:23.202017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:42455 (local address 127.31.210.195:35103)
0808 00:32:23.202091 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:23.202093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:23.202105 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:23.202196 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:23.202197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:23.202210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:23.202285 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:23.202531 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.202970 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.204233 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.204671 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.204765 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:23.204773 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:23.205063 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 00:32:23.205110 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:32:23.205466 1531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d9dc8e049048e9b82de7f2483c6798" candidate_uuid: "80f49274865446629a45b628e9cc6cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c034db3a5ee4382a8223213c9c4ae52" is_pre_election: true
I20260808 00:32:23.205468 1334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d9dc8e049048e9b82de7f2483c6798" candidate_uuid: "80f49274865446629a45b628e9cc6cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996a222826a344ae96cf01c551de7404" is_pre_election: true
I20260808 00:32:23.205691 1531 raft_consensus.cc:2454] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80f49274865446629a45b628e9cc6cbb in term 0.
I20260808 00:32:23.205688 1334 raft_consensus.cc:2454] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80f49274865446629a45b628e9cc6cbb in term 0.
I20260808 00:32:23.206043 1040 leader_election.cc:302] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f49274865446629a45b628e9cc6cbb, 996a222826a344ae96cf01c551de7404; no voters:
I20260808 00:32:23.206182 1623 raft_consensus.cc:2781] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:23.206274 1623 raft_consensus.cc:482] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:23.206311 1623 raft_consensus.cc:3037] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:23.207062 1623 raft_consensus.cc:504] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.207207 1623 leader_election.cc:288] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [CANDIDATE]: Term 1 election: Requested vote from peers 8c034db3a5ee4382a8223213c9c4ae52 (127.31.210.195:35103), 996a222826a344ae96cf01c551de7404 (127.31.210.194:38459)
I20260808 00:32:23.207392 1334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d9dc8e049048e9b82de7f2483c6798" candidate_uuid: "80f49274865446629a45b628e9cc6cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "996a222826a344ae96cf01c551de7404"
I20260808 00:32:23.207474 1334 raft_consensus.cc:3037] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:23.208271 1334 raft_consensus.cc:2454] T 66d9dc8e049048e9b82de7f2483c6798 P 996a222826a344ae96cf01c551de7404 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f49274865446629a45b628e9cc6cbb in term 1.
I20260808 00:32:23.208433 1040 leader_election.cc:302] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f49274865446629a45b628e9cc6cbb, 996a222826a344ae96cf01c551de7404; no voters:
I20260808 00:32:23.208595 1623 raft_consensus.cc:2781] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:23.208773 1623 raft_consensus.cc:686] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [term 1 LEADER]: Becoming Leader. State: Replica: 80f49274865446629a45b628e9cc6cbb, State: Running, Role: LEADER
I20260808 00:32:23.208874 1623 consensus_queue.cc:237] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } }
I20260808 00:32:23.208763 1531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d9dc8e049048e9b82de7f2483c6798" candidate_uuid: "80f49274865446629a45b628e9cc6cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c034db3a5ee4382a8223213c9c4ae52"
I20260808 00:32:23.208956 1531 raft_consensus.cc:3037] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52 [term 0 FOLLOWER]: Advancing to term 1
W20260808 00:32:23.209290 1596 tablet.cc:2369] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:23.210145 1531 raft_consensus.cc:2454] T 66d9dc8e049048e9b82de7f2483c6798 P 8c034db3a5ee4382a8223213c9c4ae52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f49274865446629a45b628e9cc6cbb in term 1.
I20260808 00:32:23.210103 925 catalog_manager.cc:5495] T 66d9dc8e049048e9b82de7f2483c6798 P 80f49274865446629a45b628e9cc6cbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80f49274865446629a45b628e9cc6cbb (127.31.210.193). New cstate: current_term: 1 leader_uuid: "80f49274865446629a45b628e9cc6cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c034db3a5ee4382a8223213c9c4ae52" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80f49274865446629a45b628e9cc6cbb" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 36615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "996a222826a344ae96cf01c551de7404" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38459 } health_report { overall_health: UNKNOWN } } }
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:23.232707 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:23.224199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.31.210.193:36615)
0808 00:32:23.224265 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:23.224267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:23.224435 (+ 168us) server_negotiation.cc:407] Connection header received
0808 00:32:23.224600 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:23.224601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:23.224622 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:23.224688 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:23.225297 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.225799 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.226658 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.226859 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.226938 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:23.229580 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:23.229595 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:23.229596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:23.229605 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:23.231289 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:23.231692 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:23.231696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:23.231697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:23.231777 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:23.232038 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:23.232039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:23.232039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:23.232192 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:23.232355 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:23.232430 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:32:23.232575 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149143, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149143, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:23.259321 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:23.251490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.31.210.193:36615)
0808 00:32:23.251554 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:23.251557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:23.251671 (+ 114us) server_negotiation.cc:407] Connection header received
0808 00:32:23.251765 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:23.251770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:23.251780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:23.251848 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:23.252461 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.253139 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.253895 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.254099 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.254177 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:23.256643 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:23.256646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:23.256647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:23.256674 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:23.258081 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:23.258485 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:23.258487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:23.258488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:23.258580 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:23.258777 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:23.258777 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:23.258777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:23.258883 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:23.259020 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:23.259141 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 00:32:23.259219 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:23.287228 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:23.280018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40260 (local address 127.31.210.193:36615)
0808 00:32:23.280097 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:23.280100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:23.280190 (+ 90us) server_negotiation.cc:407] Connection header received
0808 00:32:23.280245 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:23.280246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:23.280256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:23.280310 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:23.281054 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.281529 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.282264 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:23.282512 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:23.282585 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:23.284852 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:23.284854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:23.284854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:23.284861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:23.286309 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:23.286604 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:23.286605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:23.286606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:23.286658 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:23.286803 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:23.286803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:23.286803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:23.286884 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:23.286955 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:23.287076 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 00:32:23.287133 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 00:32:23.287994 1080 tablet_service.cc:2908] Rejecting scan request for tablet 66d9dc8e049048e9b82de7f2483c6798: Uninitialized: clean time has not yet been initialized
I20260808 00:32:23.289088 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 1008
I20260808 00:32:23.304638 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 1212
I20260808 00:32:23.313524 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 1411
I20260808 00:32:23.327483 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 887
2026-08-08T00:32:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2504 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[1647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[1647](info): set up 2 sockets
Aug 08 00:32:23 dist-test-slave-44d2 krb5kdc[1647](info): commencing operation
krb5kdc: starting...
W20260808 00:32:24.573235 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:24 dist-test-slave-44d2 krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:24.711766 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39297
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:39297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:24.803222 1664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:24.803478 1664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:24.803515 1664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:24.804749 1664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:24.804814 1664 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:32:24.804836 1664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:24.804857 1664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:24.804889 1664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:24.804913 1664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:24.806564 1664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:39297
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:24.806896 1664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:24.807147 1664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:24.812374 1671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:24.812510 1670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:24.812633 1664 server_base.cc:1034] running on GCE node
W20260808 00:32:24.812384 1673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:24.812956 1664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:24.813269 1664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:24.814467 1664 hybrid_clock.cc:648] HybridClock initialized: now 1786149144814444 us; error 41 us; skew 500 ppm
Aug 08 00:32:24 dist-test-slave-44d2 krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149144, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:24.817269 1664 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:24.817768 1664 webserver.cc:460] Webserver started at http://127.31.210.254:46139/ using document root <none> and password file <none>
I20260808 00:32:24.818049 1664 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:24.818126 1664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:24.818279 1664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:24.819383 1664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "3447df5f95244575b50ea482ac96d553"
format_stamp: "Formatted at 2026-08-08 00:32:24 on dist-test-slave-44d2"
server_key: "ab56d8afd52bd87d77e2e4964bf2eec7"
server_key_iv: "26947b5c4af3ca923f488128eff1ed36"
server_key_version: "encryptionkey@0"
I20260808 00:32:24.819775 1664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "3447df5f95244575b50ea482ac96d553"
format_stamp: "Formatted at 2026-08-08 00:32:24 on dist-test-slave-44d2"
server_key: "ab56d8afd52bd87d77e2e4964bf2eec7"
server_key_iv: "26947b5c4af3ca923f488128eff1ed36"
server_key_version: "encryptionkey@0"
I20260808 00:32:24.821251 1664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:24.822193 1685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:24.822449 1664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:24.822584 1664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "3447df5f95244575b50ea482ac96d553"
format_stamp: "Formatted at 2026-08-08 00:32:24 on dist-test-slave-44d2"
server_key: "ab56d8afd52bd87d77e2e4964bf2eec7"
server_key_iv: "26947b5c4af3ca923f488128eff1ed36"
server_key_version: "encryptionkey@0"
I20260808 00:32:24.822666 1664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:24.840447 1664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:24.842787 1664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:24.842962 1664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:24.848237 1664 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:39297
I20260808 00:32:24.848282 1774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:39297 every 8 connection(s)
I20260808 00:32:24.848716 1664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:24.849442 1775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:24.851998 1775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553: Bootstrap starting.
I20260808 00:32:24.852739 1775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:24.853075 1775 log.cc:824] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:24.853837 1775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553: No bootstrap required, opened a new log
I20260808 00:32:24.856451 1775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447df5f95244575b50ea482ac96d553" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39297 } }
I20260808 00:32:24.856680 1775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:24.856707 1775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3447df5f95244575b50ea482ac96d553, State: Initialized, Role: FOLLOWER
I20260808 00:32:24.856895 1775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447df5f95244575b50ea482ac96d553" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39297 } }
I20260808 00:32:24.857007 1775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:24.857069 1775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:24.857124 1775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:24.857756 1775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447df5f95244575b50ea482ac96d553" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39297 } }
I20260808 00:32:24.857853 1775 leader_election.cc:302] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3447df5f95244575b50ea482ac96d553; no voters:
I20260808 00:32:24.858026 1775 leader_election.cc:288] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:24.858134 1779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:24.858315 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 1664
I20260808 00:32:24.858340 1779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [term 1 LEADER]: Becoming Leader. State: Replica: 3447df5f95244575b50ea482ac96d553, State: Running, Role: LEADER
I20260808 00:32:24.858387 1775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:24.858434 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:24.858446 1779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447df5f95244575b50ea482ac96d553" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39297 } }
I20260808 00:32:24.858632 32587 external_mini_cluster.cc:1428] Setting key 817cf285ff01f2575dc8cebc61d8c4ed
I20260808 00:32:24.858946 1780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3447df5f95244575b50ea482ac96d553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447df5f95244575b50ea482ac96d553" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39297 } } }
I20260808 00:32:24.859037 1780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:24.859294 1781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3447df5f95244575b50ea482ac96d553. Latest consensus state: current_term: 1 leader_uuid: "3447df5f95244575b50ea482ac96d553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447df5f95244575b50ea482ac96d553" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39297 } } }
I20260808 00:32:24.859625 1781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:24.859791 1788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:24.860358 1788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:24.862170 1788 catalog_manager.cc:1347] Generated new cluster ID: b6435b2e74e44829abc5308aafba2b08
I20260808 00:32:24.862229 1788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:24 dist-test-slave-44d2 krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:24.872275 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:24.859360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.31.210.254:39297)
0808 00:32:24.860101 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:24.860112 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:24.860151 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:32:24.860290 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:24.860299 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:24.860364 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:24.860606 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:24.861522 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:24.863060 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:24.864191 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:24.864962 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:24.865092 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:24.868234 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:24.868239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:24.868239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:24.868252 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:24.870450 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:24.870912 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:24.870915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:24.870915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:24.871029 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:24.871283 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:24.871284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:24.871284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:24.871594 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:24.871749 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:24.871883 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:32:24.872034 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":83,"threads_started":1}
I20260808 00:32:24.879087 1788 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:24.879663 1788 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:24.885258 1788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553: Generated new TSK 0
I20260808 00:32:24.885458 1788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:24.916363 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39297
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:25.003749 1812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:25.003971 1812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:25.003996 1812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:25.005127 1812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:25.005167 1812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:25.005189 1812 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:25.005206 1812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:25.006583 1812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:39297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:25.006774 1812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:25.006958 1812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:25.012162 1819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:25.012162 1818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:25.012247 1812 server_base.cc:1034] running on GCE node
W20260808 00:32:25.012253 1821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:25.012715 1812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:25.012979 1812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:25.014191 1812 hybrid_clock.cc:648] HybridClock initialized: now 1786149145014164 us; error 64 us; skew 500 ppm
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:25.017315 1812 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:25.017911 1812 webserver.cc:460] Webserver started at http://127.31.210.193:33219/ using document root <none> and password file <none>
I20260808 00:32:25.018198 1812 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:25.018277 1812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:25.018463 1812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:25.019711 1812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "f1818484e130422c9de4cbd3b1847e5b"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "50bdf2535370a214d81c364e460d96c3"
server_key_iv: "0a615a811fd732938506ed82c28ef265"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.020140 1812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "f1818484e130422c9de4cbd3b1847e5b"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "50bdf2535370a214d81c364e460d96c3"
server_key_iv: "0a615a811fd732938506ed82c28ef265"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.021766 1812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:32:25.022691 1830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.022887 1812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:25.023093 1812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "f1818484e130422c9de4cbd3b1847e5b"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "50bdf2535370a214d81c364e460d96c3"
server_key_iv: "0a615a811fd732938506ed82c28ef265"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.023180 1812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:25.037674 1812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:25.039834 1812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:25.040084 1812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:25.040769 1812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:25.040836 1812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.040880 1812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:25.040907 1812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:25.048851 1812 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:32953
I20260808 00:32:25.048942 1986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:32953 every 8 connection(s)
I20260808 00:32:25.049314 1812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:25.051687 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.041307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:45791 (local address 127.31.210.254:39297)
0808 00:32:25.041395 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.041398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.041652 (+ 254us) server_negotiation.cc:407] Connection header received
0808 00:32:25.042407 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.042408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.042419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.042467 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.043456 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.044099 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.045270 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.045490 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.045570 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.048435 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.048437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.048437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.048450 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.050204 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.050724 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.050725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.050725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.050782 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.051019 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.051020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.051020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.051137 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.051191 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.051528 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.051617 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 00:32:25.051847 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.041276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39297 (local address 127.31.210.193:45791)
0808 00:32:25.041484 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.041535 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.042296 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.042561 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.042566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.042999 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:25.043359 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.043366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.044211 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.044213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.045161 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.045164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.045580 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.045582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.045719 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.046280 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:25.046290 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:25.048318 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:25.050369 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.050376 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.050378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.050655 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.050841 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.050841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.050841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.050934 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.051238 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:25.051246 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:25.051383 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.051496 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.051630 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260808 00:32:25.052345 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 1812
I20260808 00:32:25.052460 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:25.052497 1987 heartbeater.cc:347] Connected to a master server at 127.31.210.254:39297
I20260808 00:32:25.052615 1987 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:25.052708 32587 external_mini_cluster.cc:1428] Setting key 7a97d879795a883ef2361c646c27bce9
I20260808 00:32:25.052955 1987 heartbeater.cc:511] Master 127.31.210.254:39297 requested a full tablet report, sending...
I20260808 00:32:25.053510 1710 ts_manager.cc:194] Registered new tserver with Master: f1818484e130422c9de4cbd3b1847e5b (127.31.210.193:32953)
I20260808 00:32:25.054185 1710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:45791
I20260808 00:32:25.061021 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.053669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39297 (local address 127.31.210.193:47111)
0808 00:32:25.053818 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.053835 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.053877 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.054656 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.054657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.054911 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:25.054992 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.055003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.055972 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.055973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.056623 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.056626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.057189 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.057190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.057274 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.057794 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:25.057799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:25.058112 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:25.059770 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.059771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.059771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.060002 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.060259 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.060259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.060259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.060359 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.060746 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:25.060746 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:25.060788 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.060840 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.060926 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260808 00:32:25.061156 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.054463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:47111 (local address 127.31.210.254:39297)
0808 00:32:25.054538 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.054540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.054549 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:25.054568 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.054569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.054580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.054640 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.055108 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.055840 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.056718 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.057046 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.057127 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.058216 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.058218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.058218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.058225 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.059680 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.060111 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.060112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.060113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.060181 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.060460 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.060461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.060462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.060655 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.060742 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.060899 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.061004 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:25.084546 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39297
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:25.169915 1999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:25.170140 1999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:25.170164 1999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:25.171341 1999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:25.171386 1999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:25.171406 1999 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:25.171456 1999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:25.173025 1999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:39297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:25.173295 1999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:25.173552 1999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:25.178717 2008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:25.178735 2007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:25.179024 1999 server_base.cc:1034] running on GCE node
W20260808 00:32:25.178727 2010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:25.179283 1999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:25.179533 1999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:25.180699 1999 hybrid_clock.cc:648] HybridClock initialized: now 1786149145180694 us; error 37 us; skew 500 ppm
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:25.183449 1999 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:25.183956 1999 webserver.cc:460] Webserver started at http://127.31.210.194:37683/ using document root <none> and password file <none>
I20260808 00:32:25.184243 1999 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:25.184321 1999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:25.184537 1999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:25.185817 1999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "49c8ad63ad2f41cba6cf631f358300e6"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "48b0b3c3bc48d55ce6619083498b1b28"
server_key_iv: "6947d794a31f292033bbe9625aaa7124"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.186226 1999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "49c8ad63ad2f41cba6cf631f358300e6"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "48b0b3c3bc48d55ce6619083498b1b28"
server_key_iv: "6947d794a31f292033bbe9625aaa7124"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.187757 1999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:25.188916 2017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.189096 1999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.189218 1999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "49c8ad63ad2f41cba6cf631f358300e6"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "48b0b3c3bc48d55ce6619083498b1b28"
server_key_iv: "6947d794a31f292033bbe9625aaa7124"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.189298 1999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:25.209159 1999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:25.211640 1999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:25.211813 1999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:25.212364 1999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:25.212425 1999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.212473 1999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:25.212622 1999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.220427 1999 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:42197
I20260808 00:32:25.220531 2186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:42197 every 8 connection(s)
I20260808 00:32:25.220885 1999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:25.224174 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.213179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:39947 (local address 127.31.210.254:39297)
0808 00:32:25.213262 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.213265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.213550 (+ 285us) server_negotiation.cc:407] Connection header received
0808 00:32:25.214279 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.214281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.214296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.214367 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.215317 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.216279 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.218006 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.218351 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.218427 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.221525 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.221526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.221527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.221533 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.222728 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.223172 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.223174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.223174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.223240 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.223506 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.223507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.223508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.223633 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.223714 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.223994 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.224081 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:32:25.224365 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.213159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39297 (local address 127.31.210.194:39947)
0808 00:32:25.213368 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.213431 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.214136 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.214470 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.214476 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.214889 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:25.215215 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.215222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.216377 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.216378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.217878 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.217881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.218452 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.218454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.218621 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.219354 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:25.219365 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:25.221410 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:25.222841 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.222849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.222851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.223078 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.223312 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.223313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.223313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.223435 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.223703 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:25.223710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:25.223854 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.223970 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.224132 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260808 00:32:25.224957 2187 heartbeater.cc:347] Connected to a master server at 127.31.210.254:39297
I20260808 00:32:25.225057 2187 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:25.225265 2187 heartbeater.cc:511] Master 127.31.210.254:39297 requested a full tablet report, sending...
I20260808 00:32:25.225652 1709 ts_manager.cc:194] Registered new tserver with Master: 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197)
I20260808 00:32:25.226219 1709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:39947
I20260808 00:32:25.230578 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 1999
I20260808 00:32:25.230688 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:25.230880 32587 external_mini_cluster.cc:1428] Setting key 629a99e99662ff76cc4bbaa963a13102
I20260808 00:32:25.231952 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.226105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:48033 (local address 127.31.210.254:39297)
0808 00:32:25.226145 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.226147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.226215 (+ 68us) server_negotiation.cc:407] Connection header received
0808 00:32:25.226311 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.226312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.226323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.226370 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.226767 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.227244 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.228044 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.228258 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.228329 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.229354 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.229356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.229356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.229363 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.230696 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.231088 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.231089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.231090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.231154 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.231353 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.231354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.231354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.231551 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.231616 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.231782 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.231862 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:32:25.231982 2197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.226017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39297 (local address 127.31.210.194:48033)
0808 00:32:25.226157 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.226168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.226231 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.226390 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.226392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.226614 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:25.226689 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.226691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.227343 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.227344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.227953 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.227956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.228341 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.228342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.228424 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.228947 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:25.228953 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:25.229248 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:25.230799 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.230800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.230800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.231027 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.231214 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.231215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.231215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.231278 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.231618 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:25.231619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:25.231669 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.231755 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.231864 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:25.259891 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39297
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:25.344879 2201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:25.345129 2201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:25.345168 2201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:25.346374 2201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:25.346454 2201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:25.346493 2201 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:25.346546 2201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:25.348093 2201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:39297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:25.348372 2201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:25.348642 2201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:25.353910 2208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:25.353910 2212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:25.353911 2206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:25.354388 2201 server_base.cc:1034] running on GCE node
I20260808 00:32:25.354583 2201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:25.354800 2201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:25.355950 2201 hybrid_clock.cc:648] HybridClock initialized: now 1786149145355927 us; error 36 us; skew 500 ppm
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:25.358767 2201 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:25.359388 2201 webserver.cc:460] Webserver started at http://127.31.210.195:35715/ using document root <none> and password file <none>
I20260808 00:32:25.359618 2201 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:25.359664 2201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:25.359805 2201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:25.361052 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "00f72682032f48a2bf77b24f56817b91"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "5bcd85cd9b223ebb8d2854d5f5604168"
server_key_iv: "ad85e865668eabaca49bd38074b3a895"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.361423 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "00f72682032f48a2bf77b24f56817b91"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "5bcd85cd9b223ebb8d2854d5f5604168"
server_key_iv: "ad85e865668eabaca49bd38074b3a895"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.362821 2201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:32:25.363798 2222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.363994 2201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:25.364079 2201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "00f72682032f48a2bf77b24f56817b91"
format_stamp: "Formatted at 2026-08-08 00:32:25 on dist-test-slave-44d2"
server_key: "5bcd85cd9b223ebb8d2854d5f5604168"
server_key_iv: "ad85e865668eabaca49bd38074b3a895"
server_key_version: "encryptionkey@0"
I20260808 00:32:25.364180 2201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:25.376785 2201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:25.379340 2201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:25.379531 2201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:25.380095 2201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:25.380146 2201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.380174 2201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:25.380228 2201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:25.392539 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.380759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:38449 (local address 127.31.210.254:39297)
0808 00:32:25.380843 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.380846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.381025 (+ 179us) server_negotiation.cc:407] Connection header received
0808 00:32:25.381513 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.381515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.381530 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.381617 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.382293 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.383105 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.385075 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.385430 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.385521 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.388689 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.388700 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.388701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.388710 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.390996 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.391468 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.391469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.391470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.391535 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.391789 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.391790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.391790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.391913 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.391993 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.392327 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.392418 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:32:25.392758 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.380713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39297 (local address 127.31.210.195:38449)
0808 00:32:25.380882 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.380922 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.381412 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.381680 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.381684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.381914 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:25.382196 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.382202 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.383217 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.383219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.384849 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.384864 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.385533 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.385535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.385691 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.386414 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:25.386424 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:25.388551 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:25.391128 (+ 2577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.391136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.391138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.391372 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.391607 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.391607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.391608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.391722 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.391988 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:25.391995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:25.392185 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.392301 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.392464 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260808 00:32:25.392885 2201 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:42891
I20260808 00:32:25.392932 2395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:42891 every 8 connection(s)
I20260808 00:32:25.393373 2201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:25.394192 2396 heartbeater.cc:347] Connected to a master server at 127.31.210.254:39297
I20260808 00:32:25.394279 2396 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:25.394452 2396 heartbeater.cc:511] Master 127.31.210.254:39297 requested a full tablet report, sending...
I20260808 00:32:25.394793 1709 ts_manager.cc:194] Registered new tserver with Master: 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891)
I20260808 00:32:25.395282 1709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:38449
I20260808 00:32:25.395951 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 2201
I20260808 00:32:25.396044 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:25.396225 32587 external_mini_cluster.cc:1428] Setting key 71e7afe7b1081491a7027effdf4a6b42
I20260808 00:32:25.397071 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:32:25.400638 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.394734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:41673 (local address 127.31.210.254:39297)
0808 00:32:25.394770 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.394772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.394827 (+ 55us) server_negotiation.cc:407] Connection header received
0808 00:32:25.394918 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.394919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.394930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.395011 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.395289 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.395791 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.396810 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.397025 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.397128 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.398074 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.398075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.398075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.398083 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.399380 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.399748 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.399749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.399749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.399805 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.400021 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.400022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.400022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.400196 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.400281 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.400441 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.400548 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:32:25.400653 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.394640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39297 (local address 127.31.210.195:41673)
0808 00:32:25.394776 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.394787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.394843 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.395012 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.395013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.395162 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:25.395218 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.395219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.395910 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.395912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.396707 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.396710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.397131 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.397132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.397212 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.397670 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:25.397675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:25.397947 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:25.399498 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.399499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.399500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.399669 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.399884 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.399885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.399885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.399945 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.400274 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:25.400275 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:25.400324 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.400414 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.400550 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":8,"thread_start_us":78,"threads_started":1}
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:25.422205 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.415158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.31.210.254:39297)
0808 00:32:25.415237 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.415239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.415250 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:32:25.415279 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.415280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.415294 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.415366 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.415963 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.416450 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.417254 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.417450 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.417532 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.419882 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.419883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.419884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.419890 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.421197 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.421518 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.421519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.421519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.421574 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.421743 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.421743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.421744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.421840 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.421915 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.422074 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.422140 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:32:25.423894 1709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:25.425884 1709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:25.433476 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.428547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:42891 (local address 127.0.0.1:59654)
0808 00:32:25.428680 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.428724 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.428795 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.429402 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.429410 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.429413 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.429495 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.429498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.430945 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.430947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.432323 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.432326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.433122 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.433123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.433215 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.433228 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.433333 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.433389 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":58,"thread_start_us":89,"threads_started":1}
I20260808 00:32:25.433496 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.428827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.31.210.195:42891)
0808 00:32:25.429116 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.429125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.429133 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:25.429168 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.429169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.429181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.429314 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.429727 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.430847 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.432431 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.433030 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.433125 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.433185 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.433353 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.433392 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":165,"threads_started":1}
I20260808 00:32:25.433602 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.428744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:42197 (local address 127.0.0.1:57786)
0808 00:32:25.428935 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.428945 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.429008 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.429583 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.429584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.429586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.429641 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.429643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.430964 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.430965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.432323 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.432326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.433357 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.433358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.433425 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.433429 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.433463 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.433507 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":49,"thread_start_us":210,"threads_started":1}
I20260808 00:32:25.433847 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.429096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.31.210.193:32953)
0808 00:32:25.429323 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.429327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.429405 (+ 78us) server_negotiation.cc:407] Connection header received
0808 00:32:25.429757 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.429760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.429782 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.429905 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.430026 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.430841 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.432263 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.433153 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.433240 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.433300 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.433557 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.433598 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":128,"threads_started":1}
I20260808 00:32:25.434095 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.428766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:32953 (local address 127.0.0.1:47734)
0808 00:32:25.429318 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.429331 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.429667 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.429904 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.429905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.429908 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.429962 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.429964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.431158 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.431159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.432160 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.432163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.433342 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.433343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.433412 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.433417 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.433475 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.433508 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":1,"thread_start_us":136,"threads_started":1}
I20260808 00:32:25.434566 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.429126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57786 (local address 127.31.210.194:42197)
0808 00:32:25.429412 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.429416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.429425 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:25.429461 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.429463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.429490 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.429581 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.429709 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.430857 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.432421 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.433256 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.433806 (+ 550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.433881 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.433911 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.434452 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":96,"threads_started":1}
I20260808 00:32:25.434755 2294 tablet_service.cc:1467] Processing CreateTablet for tablet 28d65834e87643889a69fc222e11f252 (DEFAULT_TABLE table=test-table [id=2438b627304340d0a99d138c4e5b55dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:25.434828 1903 tablet_service.cc:1467] Processing CreateTablet for tablet 28d65834e87643889a69fc222e11f252 (DEFAULT_TABLE table=test-table [id=2438b627304340d0a99d138c4e5b55dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:25.435204 1903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d65834e87643889a69fc222e11f252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.435221 2294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d65834e87643889a69fc222e11f252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.435238 2086 tablet_service.cc:1467] Processing CreateTablet for tablet 28d65834e87643889a69fc222e11f252 (DEFAULT_TABLE table=test-table [id=2438b627304340d0a99d138c4e5b55dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:25.435575 2086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d65834e87643889a69fc222e11f252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.438123 2427 tablet_bootstrap.cc:492] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Bootstrap starting.
I20260808 00:32:25.439002 2427 tablet_bootstrap.cc:654] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.439188 2428 tablet_bootstrap.cc:492] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Bootstrap starting.
I20260808 00:32:25.439594 2427 log.cc:824] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:25.439586 2429 tablet_bootstrap.cc:492] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Bootstrap starting.
I20260808 00:32:25.439947 2428 tablet_bootstrap.cc:654] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.440222 2429 tablet_bootstrap.cc:654] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.440332 2428 log.cc:824] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:25.440594 2429 log.cc:824] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:25.440768 2427 tablet_bootstrap.cc:492] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: No bootstrap required, opened a new log
I20260808 00:32:25.440936 2427 ts_tablet_manager.cc:1397] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:32:25.441236 2428 tablet_bootstrap.cc:492] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: No bootstrap required, opened a new log
I20260808 00:32:25.441315 2428 ts_tablet_manager.cc:1397] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:32:25.441355 2429 tablet_bootstrap.cc:492] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: No bootstrap required, opened a new log
I20260808 00:32:25.441424 2429 ts_tablet_manager.cc:1397] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:32:25.442965 2429 raft_consensus.cc:348] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.443135 2429 raft_consensus.cc:374] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.443182 2429 raft_consensus.cc:729] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c8ad63ad2f41cba6cf631f358300e6, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.443292 2429 consensus_queue.cc:260] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.443209 2427 raft_consensus.cc:348] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.443405 2427 raft_consensus.cc:374] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.443455 2427 raft_consensus.cc:729] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f72682032f48a2bf77b24f56817b91, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.443585 2429 ts_tablet_manager.cc:1428] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:32:25.443579 2427 consensus_queue.cc:260] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.443526 2428 raft_consensus.cc:348] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.443676 2428 raft_consensus.cc:374] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.443727 2187 heartbeater.cc:503] Master 127.31.210.254:39297 was elected leader, sending a full tablet report...
I20260808 00:32:25.443729 2428 raft_consensus.cc:729] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1818484e130422c9de4cbd3b1847e5b, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.443851 2428 consensus_queue.cc:260] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.444159 1987 heartbeater.cc:503] Master 127.31.210.254:39297 was elected leader, sending a full tablet report...
I20260808 00:32:25.444231 2427 ts_tablet_manager.cc:1428] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:32:25.444358 2396 heartbeater.cc:503] Master 127.31.210.254:39297 was elected leader, sending a full tablet report...
I20260808 00:32:25.444144 2428 ts_tablet_manager.cc:1428] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:25.468278 2434 raft_consensus.cc:482] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:25.468415 2434 raft_consensus.cc:504] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.468838 2434 leader_election.cc:288] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891), 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197)
W20260808 00:32:25.471486 2189 tablet.cc:2369] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:25.472718 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.469078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:42197 (local address 127.31.210.193:54127)
0808 00:32:25.469185 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.469197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.469245 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.469453 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.469455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.469466 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.469547 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.469549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.470494 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.470495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.471723 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.471726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.472357 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.472358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.472466 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.472471 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.472563 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.472634 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 00:32:25.472723 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.469175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:54127 (local address 127.31.210.194:42197)
0808 00:32:25.469250 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.469252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.469304 (+ 52us) server_negotiation.cc:407] Connection header received
0808 00:32:25.469334 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.469335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.469347 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.469434 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.469626 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.470354 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.471855 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.472224 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.472286 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.472294 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.472576 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.472653 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:32:25.473006 2433 raft_consensus.cc:482] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:25.473031 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.469224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:48475 (local address 127.31.210.195:42891)
0808 00:32:25.469320 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.469322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.469529 (+ 207us) server_negotiation.cc:407] Connection header received
0808 00:32:25.469563 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.469564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.469572 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.469651 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.469918 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.470572 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.471827 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.472558 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.472641 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.472649 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.472849 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.472897 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 00:32:25.473100 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d65834e87643889a69fc222e11f252" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6" is_pre_election: true
I20260808 00:32:25.473106 2433 raft_consensus.cc:504] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.473279 2436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.469151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:42891 (local address 127.31.210.193:48475)
0808 00:32:25.469444 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.469454 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.469517 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.469673 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.469673 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.469675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.469806 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.469808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.470651 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.470651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.471730 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.471731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.472659 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.472661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.472747 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.472752 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.473144 (+ 392us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.473197 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":185,"threads_started":1}
I20260808 00:32:25.473273 2112 raft_consensus.cc:2454] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 0.
I20260808 00:32:25.473524 2433 leader_election.cc:288] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891), f1818484e130422c9de4cbd3b1847e5b (127.31.210.193:32953)
I20260808 00:32:25.473587 1833 leader_election.cc:302] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c8ad63ad2f41cba6cf631f358300e6, f1818484e130422c9de4cbd3b1847e5b; no voters:
I20260808 00:32:25.473698 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d65834e87643889a69fc222e11f252" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f72682032f48a2bf77b24f56817b91" is_pre_election: true
I20260808 00:32:25.473834 2325 raft_consensus.cc:2454] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 0.
I20260808 00:32:25.473869 2434 raft_consensus.cc:2781] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:25.474119 2434 raft_consensus.cc:482] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:25.474174 2434 raft_consensus.cc:3037] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:25.475160 2434 raft_consensus.cc:504] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.475472 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d65834e87643889a69fc222e11f252" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6"
I20260808 00:32:25.475472 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d65834e87643889a69fc222e11f252" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f72682032f48a2bf77b24f56817b91"
I20260808 00:32:25.475549 2112 raft_consensus.cc:3037] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:25.475558 2325 raft_consensus.cc:3037] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:25.476424 2112 raft_consensus.cc:2454] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 1.
I20260808 00:32:25.476437 2325 raft_consensus.cc:2454] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 1.
I20260808 00:32:25.476624 1833 leader_election.cc:302] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c8ad63ad2f41cba6cf631f358300e6, f1818484e130422c9de4cbd3b1847e5b; no voters:
I20260808 00:32:25.476796 2434 leader_election.cc:288] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 election: Requested vote from peers 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891), 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197)
I20260808 00:32:25.476845 2434 raft_consensus.cc:2781] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:25.476883 2434 raft_consensus.cc:686] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 1 LEADER]: Becoming Leader. State: Replica: f1818484e130422c9de4cbd3b1847e5b, State: Running, Role: LEADER
I20260808 00:32:25.476974 2434 consensus_queue.cc:237] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } }
I20260808 00:32:25.477010 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.473678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:42891 (local address 127.31.210.194:46149)
0808 00:32:25.473774 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.473786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.473827 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.474105 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.474105 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.474107 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.474169 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.474170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.474837 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.474839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.475945 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.475948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.476789 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.476790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.476867 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.476870 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.476913 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.476946 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 00:32:25.477252 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.473903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:46149 (local address 127.31.210.195:42891)
0808 00:32:25.473969 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.473970 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.473978 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:25.474001 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.474002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.474012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.474061 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.474241 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.474749 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.476091 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.476684 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.476750 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.476758 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.477159 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.477195 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:32:25.477465 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d65834e87643889a69fc222e11f252" candidate_uuid: "49c8ad63ad2f41cba6cf631f358300e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f72682032f48a2bf77b24f56817b91" is_pre_election: true
I20260808 00:32:25.477563 2325 raft_consensus.cc:2379] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49c8ad63ad2f41cba6cf631f358300e6 in current term 1: Already voted for candidate f1818484e130422c9de4cbd3b1847e5b in this term.
I20260808 00:32:25.477913 1710 catalog_manager.cc:5495] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b reported cstate change: term changed from 0 to 1, leader changed from <none> to f1818484e130422c9de4cbd3b1847e5b (127.31.210.193). New cstate: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } health_report { overall_health: HEALTHY } } }
I20260808 00:32:25.478147 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.473954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:37267 (local address 127.31.210.193:32953)
0808 00:32:25.474005 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.474007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.474132 (+ 125us) server_negotiation.cc:407] Connection header received
0808 00:32:25.474211 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.474211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.474221 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.474278 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.474449 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.475080 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.477140 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.477675 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.477724 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.477731 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.478045 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.478084 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:32:25.478158 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.473704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:32953 (local address 127.31.210.194:37267)
0808 00:32:25.474074 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.474084 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.474139 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.474306 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.474307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.474308 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.474369 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.474370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.475180 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.475181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.477016 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.477018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.477837 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.477838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.477907 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.477910 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.478026 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.478067 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":117,"threads_started":1}
I20260808 00:32:25.478871 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d65834e87643889a69fc222e11f252" candidate_uuid: "49c8ad63ad2f41cba6cf631f358300e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1818484e130422c9de4cbd3b1847e5b" is_pre_election: true
I20260808 00:32:25.479197 2021 leader_election.cc:302] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c8ad63ad2f41cba6cf631f358300e6; no voters: 00f72682032f48a2bf77b24f56817b91, f1818484e130422c9de4cbd3b1847e5b
I20260808 00:32:25.479353 2433 raft_consensus.cc:2726] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 00:32:25.495280 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.492567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.31.210.193:32953)
0808 00:32:25.492683 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.492685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.492695 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:25.492736 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.492737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.492754 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.492812 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:25.492966 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.493498 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.494334 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.494520 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.494591 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.494912 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:25.495010 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:25.495077 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.495189 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.495206 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 00:32:25.497354 2325 raft_consensus.cc:1261] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Refusing update from remote peer f1818484e130422c9de4cbd3b1847e5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:25.497704 2434 consensus_queue.cc:1035] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:25.499936 2112 raft_consensus.cc:1261] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Refusing update from remote peer f1818484e130422c9de4cbd3b1847e5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:25.500295 2445 mvcc.cc:205] Tried to move back new op lower bound from 7316066899953885184 to 7316066899874336768. Current Snapshot: MvccSnapshot[applied={T|T < 7316066899953885184}]
I20260808 00:32:25.500309 2439 consensus_queue.cc:1035] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:25.505957 1710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:25.506111 1710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:25.507031 1710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553: Sending DeleteTablet for 3 replicas of tablet 28d65834e87643889a69fc222e11f252
I20260808 00:32:25.507560 1903 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d65834e87643889a69fc222e11f252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:25 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:47734
I20260808 00:32:25.507783 2467 tablet_replica.cc:330] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: stopping tablet replica
I20260808 00:32:25.507885 2467 raft_consensus.cc:2229] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:25.507856 1710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:25.507990 1710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:25.508005 2467 raft_consensus.cc:2258] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:25.508594 2467 ts_tablet_manager.cc:1905] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:25.509563 1903 tablet_service.cc:1467] Processing CreateTablet for tablet fd489732888844e7b7f5994aea8af59d (DEFAULT_TABLE table=test-table [id=ba5dc36b8504499fa74cb676f909817d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:25.509591 2294 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d65834e87643889a69fc222e11f252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:25 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:59654
I20260808 00:32:25.509687 1903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd489732888844e7b7f5994aea8af59d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.509975 2085 tablet_service.cc:1467] Processing CreateTablet for tablet fd489732888844e7b7f5994aea8af59d (DEFAULT_TABLE table=test-table [id=ba5dc36b8504499fa74cb676f909817d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:25.509946 2086 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d65834e87643889a69fc222e11f252 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:25 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:57786
I20260808 00:32:25.510159 2085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd489732888844e7b7f5994aea8af59d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.510281 2294 tablet_service.cc:1467] Processing CreateTablet for tablet fd489732888844e7b7f5994aea8af59d (DEFAULT_TABLE table=test-table [id=ba5dc36b8504499fa74cb676f909817d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:25.510417 2294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd489732888844e7b7f5994aea8af59d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.511255 2428 tablet_bootstrap.cc:492] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Bootstrap starting.
I20260808 00:32:25.511718 2428 tablet_bootstrap.cc:654] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.512220 2429 tablet_bootstrap.cc:492] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: Bootstrap starting.
I20260808 00:32:25.512535 2427 tablet_bootstrap.cc:492] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Bootstrap starting.
I20260808 00:32:25.512662 2429 tablet_bootstrap.cc:654] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.513016 2427 tablet_bootstrap.cc:654] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.513191 2469 tablet_replica.cc:330] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: stopping tablet replica
I20260808 00:32:25.513337 2469 raft_consensus.cc:2229] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:25.513501 2469 raft_consensus.cc:2258] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:25.513870 2427 tablet_bootstrap.cc:492] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: No bootstrap required, opened a new log
I20260808 00:32:25.513931 2427 ts_tablet_manager.cc:1397] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:25.514108 2427 raft_consensus.cc:348] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.514226 2427 raft_consensus.cc:374] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.514261 2427 raft_consensus.cc:729] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f72682032f48a2bf77b24f56817b91, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.514315 2429 tablet_bootstrap.cc:492] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: No bootstrap required, opened a new log
I20260808 00:32:25.514318 2427 consensus_queue.cc:260] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.514389 2429 ts_tablet_manager.cc:1397] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:25.514428 2427 ts_tablet_manager.cc:1428] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:25.514544 2429 raft_consensus.cc:348] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.514626 2429 raft_consensus.cc:374] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.514648 2429 raft_consensus.cc:729] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c8ad63ad2f41cba6cf631f358300e6, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.514679 2428 tablet_bootstrap.cc:492] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: No bootstrap required, opened a new log
I20260808 00:32:25.514722 2428 ts_tablet_manager.cc:1397] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 00:32:25.514716 2429 consensus_queue.cc:260] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.514765 2467 ts_tablet_manager.cc:1918] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:25.514812 2429 ts_tablet_manager.cc:1428] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.514839 2467 log.cc:1196] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/28d65834e87643889a69fc222e11f252
I20260808 00:32:25.514876 2428 raft_consensus.cc:348] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.514950 2428 raft_consensus.cc:374] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.514972 2428 raft_consensus.cc:729] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1818484e130422c9de4cbd3b1847e5b, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.515023 2428 consensus_queue.cc:260] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.515102 2428 ts_tablet_manager.cc:1428] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:25.515130 2467 ts_tablet_manager.cc:1939] T 28d65834e87643889a69fc222e11f252 P f1818484e130422c9de4cbd3b1847e5b: Deleting consensus metadata
I20260808 00:32:25.515367 2468 tablet_replica.cc:330] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: stopping tablet replica
I20260808 00:32:25.515550 1688 catalog_manager.cc:4841] TS f1818484e130422c9de4cbd3b1847e5b (127.31.210.193:32953): tablet 28d65834e87643889a69fc222e11f252 (table test-table [id=2438b627304340d0a99d138c4e5b55dd]) successfully deleted
I20260808 00:32:25.515748 2468 raft_consensus.cc:2229] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:25.515872 2468 raft_consensus.cc:2258] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:25.514181 2469 ts_tablet_manager.cc:1905] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:25.516408 2468 ts_tablet_manager.cc:1905] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:25.517936 2469 ts_tablet_manager.cc:1918] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:25.518018 2469 log.cc:1196] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/28d65834e87643889a69fc222e11f252
I20260808 00:32:25.518239 2468 ts_tablet_manager.cc:1918] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:25.518316 2468 log.cc:1196] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/28d65834e87643889a69fc222e11f252
I20260808 00:32:25.518397 2469 ts_tablet_manager.cc:1939] T 28d65834e87643889a69fc222e11f252 P 49c8ad63ad2f41cba6cf631f358300e6: Deleting consensus metadata
I20260808 00:32:25.518662 2468 ts_tablet_manager.cc:1939] T 28d65834e87643889a69fc222e11f252 P 00f72682032f48a2bf77b24f56817b91: Deleting consensus metadata
I20260808 00:32:25.519009 1689 catalog_manager.cc:4841] TS 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197): tablet 28d65834e87643889a69fc222e11f252 (table test-table [id=2438b627304340d0a99d138c4e5b55dd]) successfully deleted
I20260808 00:32:25.519047 1691 catalog_manager.cc:4841] TS 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891): tablet 28d65834e87643889a69fc222e11f252 (table test-table [id=2438b627304340d0a99d138c4e5b55dd]) successfully deleted
W20260808 00:32:25.550797 1990 tablet.cc:2369] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:32:25.644176 2398 tablet.cc:2369] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:25.745157 2435 raft_consensus.cc:482] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:25.745289 2435 raft_consensus.cc:504] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.745651 2435 leader_election.cc:288] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1818484e130422c9de4cbd3b1847e5b (127.31.210.193:32953), 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197)
I20260808 00:32:25.749115 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.745858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:42197 (local address 127.31.210.195:53043)
0808 00:32:25.745902 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.745915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.745962 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.746308 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.746309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.746312 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.746404 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.746407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.747227 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.747228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.748174 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.748177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.748779 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.748780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.748880 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.748885 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.748946 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.748995 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":3}
I20260808 00:32:25.749126 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.745968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:53043 (local address 127.31.210.194:42197)
0808 00:32:25.746077 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.746080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.746093 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:32:25.746136 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.746137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.746150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.746248 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.746555 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.747131 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.748267 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.748699 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.748760 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.748768 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.748998 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.749051 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 00:32:25.749234 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.746044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:54267 (local address 127.31.210.193:32953)
0808 00:32:25.746126 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.746129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.746278 (+ 149us) server_negotiation.cc:407] Connection header received
0808 00:32:25.746314 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.746315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.746331 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.746417 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:25.746673 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.747227 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.748304 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.748813 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.748894 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.748903 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.749090 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.749141 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:32:25.749258 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.745859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:32953 (local address 127.31.210.195:54267)
0808 00:32:25.746150 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.746164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.746239 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.746446 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.746447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.746449 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:25.746578 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.746580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.747316 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.747317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.748180 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.748182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.748902 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.748903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.748998 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.749004 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.749067 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.749108 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":177,"thread_start_us":78,"threads_started":1}
I20260808 00:32:25.749377 2439 raft_consensus.cc:482] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:25.749431 2439 raft_consensus.cc:504] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.749516 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "00f72682032f48a2bf77b24f56817b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6" is_pre_election: true
I20260808 00:32:25.749599 2112 raft_consensus.cc:2454] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f72682032f48a2bf77b24f56817b91 in term 0.
I20260808 00:32:25.749615 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "00f72682032f48a2bf77b24f56817b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1818484e130422c9de4cbd3b1847e5b" is_pre_election: true
I20260808 00:32:25.749615 2439 leader_election.cc:288] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197), 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891)
I20260808 00:32:25.749718 1929 raft_consensus.cc:2454] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00f72682032f48a2bf77b24f56817b91 in term 0.
I20260808 00:32:25.749833 2226 leader_election.cc:302] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f72682032f48a2bf77b24f56817b91, 49c8ad63ad2f41cba6cf631f358300e6; no voters:
I20260808 00:32:25.749858 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f72682032f48a2bf77b24f56817b91" is_pre_election: true
I20260808 00:32:25.749926 2325 raft_consensus.cc:2454] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 0.
I20260808 00:32:25.749976 2435 raft_consensus.cc:2781] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:25.750015 2435 raft_consensus.cc:482] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:25.750036 2435 raft_consensus.cc:3037] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:25.750075 1834 leader_election.cc:302] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f72682032f48a2bf77b24f56817b91, f1818484e130422c9de4cbd3b1847e5b; no voters:
I20260808 00:32:25.750051 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6" is_pre_election: true
I20260808 00:32:25.750129 2112 raft_consensus.cc:2454] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 0.
I20260808 00:32:25.750157 2439 raft_consensus.cc:2781] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:25.750182 2439 raft_consensus.cc:482] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:25.750201 2439 raft_consensus.cc:3037] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:25.750829 2435 raft_consensus.cc:504] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.750970 2435 leader_election.cc:288] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [CANDIDATE]: Term 1 election: Requested vote from peers f1818484e130422c9de4cbd3b1847e5b (127.31.210.193:32953), 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197)
I20260808 00:32:25.751171 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "00f72682032f48a2bf77b24f56817b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6"
I20260808 00:32:25.751137 2439 raft_consensus.cc:504] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.751147 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "00f72682032f48a2bf77b24f56817b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1818484e130422c9de4cbd3b1847e5b"
I20260808 00:32:25.751259 2112 raft_consensus.cc:3037] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:25.751291 1929 raft_consensus.cc:2379] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 00f72682032f48a2bf77b24f56817b91 in current term 1: Already voted for candidate f1818484e130422c9de4cbd3b1847e5b in this term.
I20260808 00:32:25.751322 2439 leader_election.cc:288] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 election: Requested vote from peers 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197), 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891)
I20260808 00:32:25.751540 2110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6"
I20260808 00:32:25.751993 2112 raft_consensus.cc:2454] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00f72682032f48a2bf77b24f56817b91 in term 1.
I20260808 00:32:25.752234 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd489732888844e7b7f5994aea8af59d" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f72682032f48a2bf77b24f56817b91"
I20260808 00:32:25.752234 2226 leader_election.cc:302] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f72682032f48a2bf77b24f56817b91, 49c8ad63ad2f41cba6cf631f358300e6; no voters:
I20260808 00:32:25.752360 2325 raft_consensus.cc:2379] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f1818484e130422c9de4cbd3b1847e5b in current term 1: Already voted for candidate 00f72682032f48a2bf77b24f56817b91 in this term.
I20260808 00:32:25.752417 2435 raft_consensus.cc:2781] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:25.752542 1834 leader_election.cc:302] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1818484e130422c9de4cbd3b1847e5b; no voters: 00f72682032f48a2bf77b24f56817b91, 49c8ad63ad2f41cba6cf631f358300e6
I20260808 00:32:25.752563 2435 raft_consensus.cc:686] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 1 LEADER]: Becoming Leader. State: Replica: 00f72682032f48a2bf77b24f56817b91, State: Running, Role: LEADER
I20260808 00:32:25.752661 2439 raft_consensus.cc:2726] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 00:32:25.752671 2435 consensus_queue.cc:237] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.753525 1706 catalog_manager.cc:5495] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00f72682032f48a2bf77b24f56817b91 (127.31.210.195). New cstate: current_term: 1 leader_uuid: "00f72682032f48a2bf77b24f56817b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } health_report { overall_health: HEALTHY } } }
Aug 08 00:32:25 dist-test-slave-44d2 krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149144, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:25.829738 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.821599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.31.210.254:39297)
0808 00:32:25.821695 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:25.821698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:25.821826 (+ 128us) server_negotiation.cc:407] Connection header received
0808 00:32:25.821968 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:25.821969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:25.821981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:25.822060 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:25.822616 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.823134 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.823872 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.824066 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.824161 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.827103 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:25.827105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:25.827105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:25.827112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:25.828447 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.828999 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.829000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.829001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.829064 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:25.829287 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:25.829287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:25.829288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:25.829394 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:25.829455 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:25.829581 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 00:32:25.829665 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 00:32:25.829923 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:25.821508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39297 (local address 127.0.0.1:53494)
0808 00:32:25.821739 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:25.821750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:25.821851 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:25.822065 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:25.822066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:25.822446 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:25.822502 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.822504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.823235 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.823237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.823772 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:25.823773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:25.824192 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:25.824193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:25.824270 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:25.825016 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:25.825022 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:25.826989 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:25.828629 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.828637 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.828639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.828842 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.829142 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:25.829142 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:25.829142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:25.829242 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:25.829452 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:25.829459 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:25.829509 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:32:25.829561 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:32:25.829742 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":78,"thread_start_us":126,"threads_started":1}
I20260808 00:32:25.832334 1706 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:53494:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:32:25.832700 1706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:25.834194 1903 tablet_service.cc:1467] Processing CreateTablet for tablet 1174476c7e9e4e41b8069d369eed82fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ad625f5610445abae268d2a17430c64]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:25.834200 2085 tablet_service.cc:1467] Processing CreateTablet for tablet 1174476c7e9e4e41b8069d369eed82fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ad625f5610445abae268d2a17430c64]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:25.834398 2085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1174476c7e9e4e41b8069d369eed82fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.834414 1903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1174476c7e9e4e41b8069d369eed82fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.835580 2428 tablet_bootstrap.cc:492] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: Bootstrap starting.
I20260808 00:32:25.835644 2429 tablet_bootstrap.cc:492] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: Bootstrap starting.
I20260808 00:32:25.835922 2428 tablet_bootstrap.cc:654] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.835996 2429 tablet_bootstrap.cc:654] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.834178 2294 tablet_service.cc:1467] Processing CreateTablet for tablet 1174476c7e9e4e41b8069d369eed82fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ad625f5610445abae268d2a17430c64]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:25.836560 2429 tablet_bootstrap.cc:492] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: No bootstrap required, opened a new log
I20260808 00:32:25.836560 2428 tablet_bootstrap.cc:492] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: No bootstrap required, opened a new log
I20260808 00:32:25.836611 2429 ts_tablet_manager.cc:1397] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:25.836611 2428 ts_tablet_manager.cc:1397] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:25.836640 2294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1174476c7e9e4e41b8069d369eed82fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:25.836726 2428 raft_consensus.cc:348] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.836761 2429 raft_consensus.cc:348] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.836807 2428 raft_consensus.cc:374] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.836823 2429 raft_consensus.cc:374] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.836833 2428 raft_consensus.cc:729] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1818484e130422c9de4cbd3b1847e5b, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.836849 2429 raft_consensus.cc:729] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c8ad63ad2f41cba6cf631f358300e6, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.836880 2428 consensus_queue.cc:260] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.836897 2429 consensus_queue.cc:260] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.836988 2429 ts_tablet_manager.cc:1428] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.836987 2428 ts_tablet_manager.cc:1428] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:25.837108 2439 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:25.837126 2433 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:25.837188 2433 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:25.837186 2439 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:25.838272 2427 tablet_bootstrap.cc:492] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: Bootstrap starting.
I20260808 00:32:25.838690 2427 tablet_bootstrap.cc:654] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:25.839582 2427 tablet_bootstrap.cc:492] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: No bootstrap required, opened a new log
I20260808 00:32:25.839639 2427 ts_tablet_manager.cc:1397] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:25.840041 2427 raft_consensus.cc:348] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.840149 2427 raft_consensus.cc:374] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:25.840195 2427 raft_consensus.cc:729] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f72682032f48a2bf77b24f56817b91, State: Initialized, Role: FOLLOWER
I20260808 00:32:25.840250 2427 consensus_queue.cc:260] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:25.840363 2427 ts_tablet_manager.cc:1428] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:25.840453 2435 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:25.840569 2435 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:26.090844 2439 raft_consensus.cc:482] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:26.090966 2439 raft_consensus.cc:504] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:26.091248 2439 leader_election.cc:288] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197), 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891)
I20260808 00:32:26.091574 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174476c7e9e4e41b8069d369eed82fb" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f72682032f48a2bf77b24f56817b91" is_pre_election: true
I20260808 00:32:26.091706 2325 raft_consensus.cc:2454] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 0.
I20260808 00:32:26.091612 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174476c7e9e4e41b8069d369eed82fb" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6" is_pre_election: true
I20260808 00:32:26.091761 2112 raft_consensus.cc:2454] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 0.
I20260808 00:32:26.091915 1834 leader_election.cc:302] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f72682032f48a2bf77b24f56817b91, f1818484e130422c9de4cbd3b1847e5b; no voters:
I20260808 00:32:26.092132 2439 raft_consensus.cc:2781] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:26.092203 2439 raft_consensus.cc:482] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:26.092221 2439 raft_consensus.cc:3037] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:26.093138 2439 raft_consensus.cc:504] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:26.093291 2439 leader_election.cc:288] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 election: Requested vote from peers 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197), 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891)
I20260808 00:32:26.093516 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174476c7e9e4e41b8069d369eed82fb" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f72682032f48a2bf77b24f56817b91"
I20260808 00:32:26.093696 2325 raft_consensus.cc:3037] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:26.093672 2112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174476c7e9e4e41b8069d369eed82fb" candidate_uuid: "f1818484e130422c9de4cbd3b1847e5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c8ad63ad2f41cba6cf631f358300e6"
I20260808 00:32:26.093815 2112 raft_consensus.cc:3037] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:26.094445 2325 raft_consensus.cc:2454] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 1.
I20260808 00:32:26.094666 1834 leader_election.cc:302] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f72682032f48a2bf77b24f56817b91, f1818484e130422c9de4cbd3b1847e5b; no voters:
I20260808 00:32:26.094765 2112 raft_consensus.cc:2454] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1818484e130422c9de4cbd3b1847e5b in term 1.
I20260808 00:32:26.094786 2439 raft_consensus.cc:2781] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:26.094894 2439 raft_consensus.cc:686] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [term 1 LEADER]: Becoming Leader. State: Replica: f1818484e130422c9de4cbd3b1847e5b, State: Running, Role: LEADER
I20260808 00:32:26.094950 2439 consensus_queue.cc:237] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } }
I20260808 00:32:26.095232 2492 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: TxnStatusTablet state changed. Reason: New leader f1818484e130422c9de4cbd3b1847e5b. Latest consensus state: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:26.095316 2492 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:26.095620 2495 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:32:26.095613 1709 catalog_manager.cc:5495] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b reported cstate change: term changed from 0 to 1, leader changed from <none> to f1818484e130422c9de4cbd3b1847e5b (127.31.210.193). New cstate: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:26.095762 2495 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:32:26.126308 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.123345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:32953 (local address 127.0.0.1:47746)
0808 00:32:26.123440 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:26.123455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:26.123516 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:26.123687 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:26.123687 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:26.123688 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:26.123740 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:26.123743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.124734 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.124735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:26.125371 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:26.125373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.125856 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.125858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:26.125947 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.125963 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:26.126172 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:26.126176 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:26.126208 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 00:32:26.126248 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 00:32:26.126317 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.123456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47746 (local address 127.31.210.193:32953)
0808 00:32:26.123537 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:26.123540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:26.123574 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:32:26.123602 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:26.123603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:26.123615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:26.123669 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:26.123803 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.124611 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.125452 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.125769 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.125843 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.126075 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:26.126111 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:26.126171 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:26.126211 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 00:32:26.126233 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 00:32:26.127583 2325 raft_consensus.cc:1261] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Refusing update from remote peer f1818484e130422c9de4cbd3b1847e5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:26.127704 2112 raft_consensus.cc:1261] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Refusing update from remote peer f1818484e130422c9de4cbd3b1847e5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:26.127872 2439 consensus_queue.cc:1035] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:26.127969 2492 consensus_queue.cc:1035] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:26.128557 2435 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: TxnStatusTablet state changed. Reason: New leader f1818484e130422c9de4cbd3b1847e5b. Latest consensus state: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:26.128664 2435 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:26.129274 2500 mvcc.cc:205] Tried to move back new op lower bound from 7316066902536880128 to 7316066902405394432. Current Snapshot: MvccSnapshot[applied={T|T < 7316066902536880128}]
I20260808 00:32:26.129406 2433 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: TxnStatusTablet state changed. Reason: New leader f1818484e130422c9de4cbd3b1847e5b. Latest consensus state: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:26.129496 2433 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:26.129722 2492 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:26.129823 2492 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:26.130133 2435 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:26.130227 2435 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P 00f72682032f48a2bf77b24f56817b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:26.133167 2492 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:26.133268 2492 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P f1818484e130422c9de4cbd3b1847e5b: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:26.133771 2433 tablet_replica.cc:439] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1818484e130422c9de4cbd3b1847e5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 } } peers { permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 } } peers { permanent_uuid: "00f72682032f48a2bf77b24f56817b91" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 42891 } } }
I20260808 00:32:26.133898 2433 tablet_replica.cc:442] T 1174476c7e9e4e41b8069d369eed82fb P 49c8ad63ad2f41cba6cf631f358300e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:26.135157 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.131807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.31.210.195:42891)
0808 00:32:26.131921 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:26.131923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:26.131933 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:26.132022 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:26.132023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:26.132048 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:26.132134 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:26.132242 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.133339 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.134425 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.134630 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.134685 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.134860 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:26.134962 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:26.135068 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:26.135083 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 00:32:26.135101 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
I20260808 00:32:26.140712 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.137727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:32953 (local address 127.31.210.195:39535)
0808 00:32:26.137918 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:26.138002 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:26.138062 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:26.138190 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:26.138191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:26.138192 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:26.138239 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:26.138240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.139105 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.139105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:26.139757 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:26.139759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.140240 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.140241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:26.140291 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.140305 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:26.140546 (+ 241us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:26.140549 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:26.140603 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:32:26.140643 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":121,"threads_started":1}
I20260808 00:32:26.140707 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.137799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:39535 (local address 127.31.210.193:32953)
0808 00:32:26.137845 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:26.137847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:26.138072 (+ 225us) server_negotiation.cc:407] Connection header received
0808 00:32:26.138103 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:26.138104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:26.138114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:26.138178 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:26.138304 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.139017 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.139849 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.140148 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.140218 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.140401 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:26.140434 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:26.140519 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:26.140607 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:32:26.140630 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:32:26.143296 2112 raft_consensus.cc:1261] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Refusing update from remote peer 00f72682032f48a2bf77b24f56817b91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:26.143380 1929 raft_consensus.cc:1261] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Refusing update from remote peer 00f72682032f48a2bf77b24f56817b91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:26.143638 2477 consensus_queue.cc:1035] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c8ad63ad2f41cba6cf631f358300e6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:26.143751 2435 consensus_queue.cc:1035] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1818484e130422c9de4cbd3b1847e5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 32953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:26.153992 2524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.150824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:42891 (local address 127.31.210.193:46391)
0808 00:32:26.150999 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:26.151011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:26.151105 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:26.151244 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:26.151245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:26.151246 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:26.151315 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:26.151317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.152104 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.152105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:26.153005 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:26.153008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.153447 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.153448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:26.153509 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.153526 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:26.153804 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:26.153807 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:26.153860 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:32:26.153906 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260808 00:32:26.153975 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.150864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:46391 (local address 127.31.210.195:42891)
0808 00:32:26.150909 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:26.150911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:26.151091 (+ 180us) server_negotiation.cc:407] Connection header received
0808 00:32:26.151164 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:26.151165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:26.151177 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:26.151239 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:26.151400 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.151982 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.153133 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.153361 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.153447 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.153670 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:26.153729 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:26.153801 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:26.153861 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 00:32:26.153885 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:32:26.165335 1706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:26.165467 1706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:26.166221 1706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3447df5f95244575b50ea482ac96d553: Sending DeleteTablet for 3 replicas of tablet fd489732888844e7b7f5994aea8af59d
I20260808 00:32:26.166522 1903 tablet_service.cc:1514] Processing DeleteTablet for tablet fd489732888844e7b7f5994aea8af59d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:26 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:47734
I20260808 00:32:26.166652 2085 tablet_service.cc:1514] Processing DeleteTablet for tablet fd489732888844e7b7f5994aea8af59d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:26 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:57786
I20260808 00:32:26.166888 2530 tablet_replica.cc:330] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: stopping tablet replica
I20260808 00:32:26.166960 2530 raft_consensus.cc:2229] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:26.167009 2530 raft_consensus.cc:2258] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:26.167359 2530 ts_tablet_manager.cc:1905] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:26.167734 2294 tablet_service.cc:1514] Processing DeleteTablet for tablet fd489732888844e7b7f5994aea8af59d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:26 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:59654
I20260808 00:32:26.168272 2533 tablet_replica.cc:330] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: stopping tablet replica
I20260808 00:32:26.168346 2533 raft_consensus.cc:2229] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:26.168435 2533 raft_consensus.cc:2258] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:26.169451 2533 ts_tablet_manager.cc:1905] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:26.169596 2530 ts_tablet_manager.cc:1918] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:26.169672 2530 log.cc:1196] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/fd489732888844e7b7f5994aea8af59d
I20260808 00:32:26.169968 2530 ts_tablet_manager.cc:1939] T fd489732888844e7b7f5994aea8af59d P 49c8ad63ad2f41cba6cf631f358300e6: Deleting consensus metadata
I20260808 00:32:26.170302 1689 catalog_manager.cc:4841] TS 49c8ad63ad2f41cba6cf631f358300e6 (127.31.210.194:42197): tablet fd489732888844e7b7f5994aea8af59d (table test-table [id=ba5dc36b8504499fa74cb676f909817d]) successfully deleted
I20260808 00:32:26.171034 2533 ts_tablet_manager.cc:1918] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:26.171099 2533 log.cc:1196] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/fd489732888844e7b7f5994aea8af59d
I20260808 00:32:26.171341 2533 ts_tablet_manager.cc:1939] T fd489732888844e7b7f5994aea8af59d P 00f72682032f48a2bf77b24f56817b91: Deleting consensus metadata
I20260808 00:32:26.171684 1691 catalog_manager.cc:4841] TS 00f72682032f48a2bf77b24f56817b91 (127.31.210.195:42891): tablet fd489732888844e7b7f5994aea8af59d (table test-table [id=ba5dc36b8504499fa74cb676f909817d]) successfully deleted
I20260808 00:32:26.172638 2529 tablet_replica.cc:330] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: stopping tablet replica
I20260808 00:32:26.172719 2529 raft_consensus.cc:2229] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:26.172770 2529 raft_consensus.cc:2258] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:26.173099 2529 ts_tablet_manager.cc:1905] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:26.174782 2529 ts_tablet_manager.cc:1918] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:26.174901 2529 log.cc:1196] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/fd489732888844e7b7f5994aea8af59d
I20260808 00:32:26.175204 2529 ts_tablet_manager.cc:1939] T fd489732888844e7b7f5994aea8af59d P f1818484e130422c9de4cbd3b1847e5b: Deleting consensus metadata
I20260808 00:32:26.175583 1688 catalog_manager.cc:4841] TS f1818484e130422c9de4cbd3b1847e5b (127.31.210.193:32953): tablet fd489732888844e7b7f5994aea8af59d (table test-table [id=ba5dc36b8504499fa74cb676f909817d]) successfully deleted
Aug 08 00:32:26 dist-test-slave-44d2 krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:26.184180 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.168173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.31.210.193:32953)
0808 00:32:26.168222 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:26.168224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:26.173006 (+ 4782us) server_negotiation.cc:407] Connection header received
0808 00:32:26.173138 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:26.173139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:26.173151 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:26.173185 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:26.174043 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.174868 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.176081 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.176438 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.176506 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.180458 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:26.180467 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:26.180468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:26.180476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:26.182726 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:26.183175 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:26.183179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:26.183180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:26.183253 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:26.183513 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:26.183514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:26.183514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:26.183712 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:26.183863 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:26.183974 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 00:32:26.184072 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:32:26.184551 1974 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47748
I20260808 00:32:26.194310 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:26.185701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.31.210.254:39297)
0808 00:32:26.185757 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:26.185759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:26.186021 (+ 262us) server_negotiation.cc:407] Connection header received
0808 00:32:26.186099 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:26.186100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:26.186114 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:26.186178 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:26.186853 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.187606 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.188829 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:26.189124 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:26.189207 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:26.190761 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:26.190763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:26.190764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:26.190771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:26.192873 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:26.193336 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:26.193337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:26.193337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:26.193411 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:26.193655 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:26.193656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:26.193656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:26.193799 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:26.193946 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:26.194129 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 00:32:26.194213 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 00:32:26.194604 1706 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53506
I20260808 00:32:26.195528 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 1812
I20260808 00:32:26.214399 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 1999
I20260808 00:32:26.225637 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 2201
I20260808 00:32:26.235808 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 1664
2026-08-08T00:32:26Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2905 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:26 dist-test-slave-44d2 krb5kdc[2540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:26 dist-test-slave-44d2 krb5kdc[2540](info): set up 2 sockets
Aug 08 00:32:26 dist-test-slave-44d2 krb5kdc[2540](info): commencing operation
krb5kdc: starting...
W20260808 00:32:27.464143 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:27 dist-test-slave-44d2 krb5kdc[2540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:27.600272 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43129
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:43129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:27.684741 2555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:27.685005 2555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:27.685045 2555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:27.686210 2555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:27.686285 2555 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:32:27.686308 2555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:27.686331 2555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:27.686362 2555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:27.686386 2555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:27.687889 2555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:43129
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:27.688184 2555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:27.688416 2555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:27.693682 2570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:27.693773 2567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:27.693697 2565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:27.694080 2555 server_base.cc:1034] running on GCE node
I20260808 00:32:27.694356 2555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:27.694738 2555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:27.695964 2555 hybrid_clock.cc:648] HybridClock initialized: now 1786149147695913 us; error 81 us; skew 500 ppm
Aug 08 00:32:27 dist-test-slave-44d2 krb5kdc[2540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149147, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:27.703709 2555 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:27.704243 2555 webserver.cc:460] Webserver started at http://127.31.210.254:33701/ using document root <none> and password file <none>
I20260808 00:32:27.704536 2555 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:27.704618 2555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:27.704785 2555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:27.705885 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "34c092bf9f74480b98637511c4992205"
format_stamp: "Formatted at 2026-08-08 00:32:27 on dist-test-slave-44d2"
server_key: "aa888de187f148392e649facc60dff14"
server_key_iv: "5edcb56dbc330d879c201820b3a4bfa5"
server_key_version: "encryptionkey@0"
I20260808 00:32:27.706339 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "34c092bf9f74480b98637511c4992205"
format_stamp: "Formatted at 2026-08-08 00:32:27 on dist-test-slave-44d2"
server_key: "aa888de187f148392e649facc60dff14"
server_key_iv: "5edcb56dbc330d879c201820b3a4bfa5"
server_key_version: "encryptionkey@0"
I20260808 00:32:27.707839 2555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:27.708775 2578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:27.709128 2555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:27.709272 2555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "34c092bf9f74480b98637511c4992205"
format_stamp: "Formatted at 2026-08-08 00:32:27 on dist-test-slave-44d2"
server_key: "aa888de187f148392e649facc60dff14"
server_key_iv: "5edcb56dbc330d879c201820b3a4bfa5"
server_key_version: "encryptionkey@0"
I20260808 00:32:27.709363 2555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:27.717581 2555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:27.720201 2555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:27.720391 2555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:27.726313 2555 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:43129
I20260808 00:32:27.726428 2668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:43129 every 8 connection(s)
I20260808 00:32:27.726776 2555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:27.727443 2669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:27.729974 2669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Bootstrap starting.
I20260808 00:32:27.730613 2669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:27.730927 2669 log.cc:824] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:27.731828 2669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: No bootstrap required, opened a new log
I20260808 00:32:27.734213 2669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c092bf9f74480b98637511c4992205" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43129 } }
I20260808 00:32:27.734371 2669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:27.734395 2669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c092bf9f74480b98637511c4992205, State: Initialized, Role: FOLLOWER
I20260808 00:32:27.734561 2669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c092bf9f74480b98637511c4992205" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43129 } }
I20260808 00:32:27.734648 2669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:27.734673 2669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:27.734748 2669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:27.735351 2669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c092bf9f74480b98637511c4992205" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43129 } }
I20260808 00:32:27.735471 2669 leader_election.cc:302] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c092bf9f74480b98637511c4992205; no voters:
I20260808 00:32:27.735675 2669 leader_election.cc:288] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:27.735736 2675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:27.735996 2669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:27.736133 2675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [term 1 LEADER]: Becoming Leader. State: Replica: 34c092bf9f74480b98637511c4992205, State: Running, Role: LEADER
I20260808 00:32:27.736418 2675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c092bf9f74480b98637511c4992205" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43129 } }
I20260808 00:32:27.736618 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 2555
I20260808 00:32:27.736723 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:27.736922 32587 external_mini_cluster.cc:1428] Setting key 80a2a7cbaddb6213044eb586ec27d53e
I20260808 00:32:27.736910 2681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c092bf9f74480b98637511c4992205. Latest consensus state: current_term: 1 leader_uuid: "34c092bf9f74480b98637511c4992205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c092bf9f74480b98637511c4992205" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43129 } } }
I20260808 00:32:27.737004 2681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:27.737380 2679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c092bf9f74480b98637511c4992205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c092bf9f74480b98637511c4992205" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43129 } } }
I20260808 00:32:27.737474 2679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205 [sys.catalog]: This master's current role is: LEADER
W20260808 00:32:27.738196 2701 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:32:27.738291 2701 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:32:27.738933 2690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:27.739153 2690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:27.740634 2690 catalog_manager.cc:1347] Generated new cluster ID: 10c6b18e653546529db27d5e2cbbe964
I20260808 00:32:27.740687 2690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:27 dist-test-slave-44d2 krb5kdc[2540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:27.750810 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:27.737513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.31.210.254:43129)
0808 00:32:27.737687 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:27.737700 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:27.737764 (+ 64us) server_negotiation.cc:407] Connection header received
0808 00:32:27.737921 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:27.737930 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:27.738008 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:27.738177 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:27.738986 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.740572 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.741824 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.742698 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.742839 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:27.746482 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:27.746488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:27.746488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:27.746503 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:27.748856 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:27.749331 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:27.749334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:27.749335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:27.749478 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:27.749729 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:27.749730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:27.749730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:27.750071 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:27.750235 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:27.750392 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 00:32:27.750555 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260808 00:32:27.752355 2690 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:27.753211 2690 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:27.760910 2690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Generated new TSK 0
I20260808 00:32:27.761121 2690 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:27.795792 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43129
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:27.890074 2707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:27.890285 2707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:27.890311 2707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:27.891467 2707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:27.891512 2707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:27.891534 2707 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:32:27.891583 2707 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:27.891610 2707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:27.893173 2707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:43129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:27.893437 2707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:27.893676 2707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:27.899063 2719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:27.899063 2721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:27.899122 2718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:27.899454 2707 server_base.cc:1034] running on GCE node
I20260808 00:32:27.899670 2707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:27.899933 2707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:27.901113 2707 hybrid_clock.cc:648] HybridClock initialized: now 1786149147901078 us; error 46 us; skew 500 ppm
Aug 08 00:32:27 dist-test-slave-44d2 krb5kdc[2540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149147, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:27.903934 2707 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:27.904520 2707 webserver.cc:460] Webserver started at http://127.31.210.193:33517/ using document root <none> and password file <none>
I20260808 00:32:27.904790 2707 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:27.904838 2707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:27.905015 2707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:27.906378 2707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "42a117c33be64331b0eb35e9d6e309b5"
format_stamp: "Formatted at 2026-08-08 00:32:27 on dist-test-slave-44d2"
server_key: "9c636ef35020aad6bb0bdbac87648bdf"
server_key_iv: "20734d7419220e084e9ad002023fef56"
server_key_version: "encryptionkey@0"
I20260808 00:32:27.906956 2707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "42a117c33be64331b0eb35e9d6e309b5"
format_stamp: "Formatted at 2026-08-08 00:32:27 on dist-test-slave-44d2"
server_key: "9c636ef35020aad6bb0bdbac87648bdf"
server_key_iv: "20734d7419220e084e9ad002023fef56"
server_key_version: "encryptionkey@0"
I20260808 00:32:27.908752 2707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:32:27.909696 2731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:27.909896 2707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:27.910014 2707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "42a117c33be64331b0eb35e9d6e309b5"
format_stamp: "Formatted at 2026-08-08 00:32:27 on dist-test-slave-44d2"
server_key: "9c636ef35020aad6bb0bdbac87648bdf"
server_key_iv: "20734d7419220e084e9ad002023fef56"
server_key_version: "encryptionkey@0"
I20260808 00:32:27.910110 2707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:27.933425 2707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:27.936025 2707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:27.936327 2707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:27.937026 2707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:27.937073 2707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:27.937098 2707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:27.937137 2707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:32:27 dist-test-slave-44d2 krb5kdc[2540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149147, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:27.946290 2707 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:38099
I20260808 00:32:27.946373 2899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:38099 every 8 connection(s)
I20260808 00:32:27.946841 2707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:27.949085 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:27.937597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:53719 (local address 127.31.210.254:43129)
0808 00:32:27.937680 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:27.937683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:27.937880 (+ 197us) server_negotiation.cc:407] Connection header received
0808 00:32:27.938429 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:27.938431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:27.938451 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:27.938588 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:27.939439 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.940474 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.942007 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.942413 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.942509 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:27.945881 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:27.945884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:27.945884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:27.945894 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:27.947761 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:27.948179 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:27.948180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:27.948181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:27.948241 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:27.948453 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:27.948454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:27.948454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:27.948632 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:27.948686 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:27.948946 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 00:32:27.949020 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:32:27.949397 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:27.937534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43129 (local address 127.31.210.193:53719)
0808 00:32:27.937707 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:27.937751 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:27.938309 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:27.938642 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:27.938646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:27.939012 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:27.939310 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:27.939318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.940702 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.940703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:27.941914 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:27.941917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.942532 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.942534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:27.942705 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:27.943377 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:27.943388 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:27.945769 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:27.947848 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:27.947853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:27.947855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:27.948104 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:27.948301 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:27.948301 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:27.948301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:27.948399 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:27.948688 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:27.948692 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:27.948828 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 00:32:27.948917 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:32:27.949127 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260808 00:32:27.950068 2901 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43129
I20260808 00:32:27.950156 2901 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:27.950343 2901 heartbeater.cc:511] Master 127.31.210.254:43129 requested a full tablet report, sending...
I20260808 00:32:27.950922 2599 ts_manager.cc:194] Registered new tserver with Master: 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099)
I20260808 00:32:27.951707 2599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:53719
I20260808 00:32:27.953454 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 2707
I20260808 00:32:27.953630 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:27.953891 32587 external_mini_cluster.cc:1428] Setting key b64944d97a0a80fc9121f186ad4ea1f5
I20260808 00:32:27.958127 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:27.951375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:40445 (local address 127.31.210.254:43129)
0808 00:32:27.951426 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:27.951428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:27.951565 (+ 137us) server_negotiation.cc:407] Connection header received
0808 00:32:27.951596 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:27.951597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:27.951607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:27.951677 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:27.952061 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.952623 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.953423 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.953815 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.953888 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:27.955202 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:27.955203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:27.955203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:27.955210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:27.956809 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:27.957278 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:27.957279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:27.957279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:27.957338 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:27.957589 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:27.957590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:27.957591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:27.957695 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:27.957753 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:27.957951 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 00:32:27.958064 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":25}
I20260808 00:32:27.958226 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:27.951308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43129 (local address 127.31.210.193:40445)
0808 00:32:27.951489 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:27.951504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:27.951558 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:27.951681 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:27.951681 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:27.951909 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:27.951986 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:27.951988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.952695 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.952696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:27.953306 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:27.953308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:27.953903 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:27.953904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:27.954010 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:27.954586 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:27.954593 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:27.955105 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:27.956905 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:27.956907 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:27.956907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:27.957178 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:27.957433 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:27.957433 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:27.957434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:27.957524 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:27.957768 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:27.957769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:27.957827 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 00:32:27.957924 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 00:32:27.958067 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:27.984009 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43129
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:28.067628 2917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:28.067843 2917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:28.067870 2917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:28.069525 2917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:28.069595 2917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:28.069620 2917 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:32:28.069634 2917 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:28.069654 2917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:28.071354 2917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:43129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:28.071633 2917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:28.071909 2917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:28.077422 2927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:28.077378 2924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:28.077445 2930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:28.078027 2917 server_base.cc:1034] running on GCE node
I20260808 00:32:28.078295 2917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:28.078562 2917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:28.079746 2917 hybrid_clock.cc:648] HybridClock initialized: now 1786149148079727 us; error 29 us; skew 500 ppm
Aug 08 00:32:28 dist-test-slave-44d2 krb5kdc[2540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149148, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:28.082654 2917 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:28.083137 2917 webserver.cc:460] Webserver started at http://127.31.210.194:35321/ using document root <none> and password file <none>
I20260808 00:32:28.083365 2917 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:28.083415 2917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:28.083511 2917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:28.084715 2917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "016bfba080a042e2b38f6db8d0ac1616"
format_stamp: "Formatted at 2026-08-08 00:32:28 on dist-test-slave-44d2"
server_key: "50093c553d7528ccdb9e5197141f35c2"
server_key_iv: "ef923dda695d91a2cfb0497351fd00ee"
server_key_version: "encryptionkey@0"
I20260808 00:32:28.085089 2917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "016bfba080a042e2b38f6db8d0ac1616"
format_stamp: "Formatted at 2026-08-08 00:32:28 on dist-test-slave-44d2"
server_key: "50093c553d7528ccdb9e5197141f35c2"
server_key_iv: "ef923dda695d91a2cfb0497351fd00ee"
server_key_version: "encryptionkey@0"
I20260808 00:32:28.086584 2917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:28.087399 2937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.087597 2917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:28.087692 2917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "016bfba080a042e2b38f6db8d0ac1616"
format_stamp: "Formatted at 2026-08-08 00:32:28 on dist-test-slave-44d2"
server_key: "50093c553d7528ccdb9e5197141f35c2"
server_key_iv: "ef923dda695d91a2cfb0497351fd00ee"
server_key_version: "encryptionkey@0"
I20260808 00:32:28.087766 2917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:28.098521 2917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:28.100575 2917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:28.100733 2917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:28.101248 2917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:28.101290 2917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.101313 2917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:28.101356 2917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.108985 2917 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:34791
Aug 08 00:32:28 dist-test-slave-44d2 krb5kdc[2540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149148, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:28.109054 3095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:34791 every 8 connection(s)
I20260808 00:32:28.109388 2917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:28.109864 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 2917
I20260808 00:32:28.109987 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:28.110216 32587 external_mini_cluster.cc:1428] Setting key 7a23167f175f02e6f1b47bbd3e351fe8
I20260808 00:32:28.113092 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.101821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:47689 (local address 127.31.210.254:43129)
0808 00:32:28.101880 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.101882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.102160 (+ 278us) server_negotiation.cc:407] Connection header received
0808 00:32:28.102782 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.102785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.102802 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.102882 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:28.103793 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.104666 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.106425 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.106768 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.106854 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.109606 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:28.109608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:28.109608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:28.109616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:28.111476 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.111880 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.111881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.111882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.111951 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.112265 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.112266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.112266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.112417 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:28.112510 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.112839 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.112949 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:32:28.113308 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.101808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43129 (local address 127.31.210.194:47689)
0808 00:32:28.102033 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.102069 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.102650 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.103022 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.103029 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.103388 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:28.103674 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.103682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.104822 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.104823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.106302 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.106306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.106864 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.106865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.106999 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.107552 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:28.107564 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:28.109500 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:28.111562 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.111569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.111570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.111801 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.112085 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.112086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.112086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.112195 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.112533 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:28.112539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:28.112696 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.112785 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.113012 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":100,"threads_started":1}
I20260808 00:32:28.113927 3096 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43129
I20260808 00:32:28.114034 3096 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:28.114277 3096 heartbeater.cc:511] Master 127.31.210.254:43129 requested a full tablet report, sending...
I20260808 00:32:28.115118 2598 ts_manager.cc:194] Registered new tserver with Master: 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791)
I20260808 00:32:28.115540 2598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:47689
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
I20260808 00:32:28.123888 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.115024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43129 (local address 127.31.210.194:58059)
0808 00:32:28.116662 (+ 1638us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.116718 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.116808 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.117152 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.117153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.117435 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:28.117525 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.117527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.118245 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.118247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.119272 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.119276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.119651 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.119653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.119751 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.120292 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:28.120299 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:28.120875 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:28.122560 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.122562 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.122563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.122811 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.123087 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.123088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.123088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.123154 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.123618 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:28.123619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:28.123664 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.123727 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.123816 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1592,"thread_start_us":70,"threads_started":1}
I20260808 00:32:28.123922 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.115098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:58059 (local address 127.31.210.254:43129)
0808 00:32:28.115142 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.115143 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.116808 (+ 1665us) server_negotiation.cc:407] Connection header received
0808 00:32:28.117041 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.117042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.117054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.117114 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:28.117642 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.118114 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.119376 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.119563 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.119618 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.120971 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:28.120973 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:28.120974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:28.120981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:28.122449 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.122931 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.122933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.122933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.123003 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.123223 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.123224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.123224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.123500 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:28.123611 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.123751 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.123854 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:28.143319 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43129
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:28.228796 3110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:28.228988 3110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:28.229012 3110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:28.230157 3110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:28.230202 3110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:28.230223 3110 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:32:28.230275 3110 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:28.230304 3110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:28.231750 3110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:43129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:28.232004 3110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:28.232260 3110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:28.237356 3117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:28.237478 3121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:28.237368 3116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:28.237744 3110 server_base.cc:1034] running on GCE node
I20260808 00:32:28.237953 3110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:28.238197 3110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:28.239338 3110 hybrid_clock.cc:648] HybridClock initialized: now 1786149148239324 us; error 28 us; skew 500 ppm
Aug 08 00:32:28 dist-test-slave-44d2 krb5kdc[2540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149148, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:28.242097 3110 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:28.242568 3110 webserver.cc:460] Webserver started at http://127.31.210.195:39441/ using document root <none> and password file <none>
I20260808 00:32:28.242818 3110 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:28.242864 3110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:28.243036 3110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:28.244087 3110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "46a836c8948e49e2a1033839ae41f048"
format_stamp: "Formatted at 2026-08-08 00:32:28 on dist-test-slave-44d2"
server_key: "ec309b3f6b9bbb041a79712582c2dd4b"
server_key_iv: "54314f468bc99e533d3b45fbd885d271"
server_key_version: "encryptionkey@0"
I20260808 00:32:28.244445 3110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "46a836c8948e49e2a1033839ae41f048"
format_stamp: "Formatted at 2026-08-08 00:32:28 on dist-test-slave-44d2"
server_key: "ec309b3f6b9bbb041a79712582c2dd4b"
server_key_iv: "54314f468bc99e533d3b45fbd885d271"
server_key_version: "encryptionkey@0"
I20260808 00:32:28.245853 3110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:28.246659 3129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.246847 3110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:28.246942 3110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "46a836c8948e49e2a1033839ae41f048"
format_stamp: "Formatted at 2026-08-08 00:32:28 on dist-test-slave-44d2"
server_key: "ec309b3f6b9bbb041a79712582c2dd4b"
server_key_iv: "54314f468bc99e533d3b45fbd885d271"
server_key_version: "encryptionkey@0"
I20260808 00:32:28.247064 3110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:28.276796 3110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:28.279062 3110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:28.279193 3110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:28.279917 3110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:28.279973 3110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.280018 3110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:28.280043 3110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:32:28 dist-test-slave-44d2 krb5kdc[2540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149148, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:28.288334 3110 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:40661
I20260808 00:32:28.288750 3110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:28.289479 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 3110
I20260808 00:32:28.289580 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:28.289811 32587 external_mini_cluster.cc:1428] Setting key c61ab11541b1912e30535b0fa8e8f761
I20260808 00:32:28.289669 3306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:40661 every 8 connection(s)
I20260808 00:32:28.290961 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.280419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:58349 (local address 127.31.210.254:43129)
0808 00:32:28.280526 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.280529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.280732 (+ 203us) server_negotiation.cc:407] Connection header received
0808 00:32:28.281252 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.281253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.281270 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.281356 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:28.282084 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.282934 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.284122 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.284334 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.284426 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.287189 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:28.287191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:28.287191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:28.287201 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:28.289018 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.289606 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.289607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.289608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.289687 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.290160 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.290161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.290161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.290303 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:28.290389 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.290777 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.290870 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:32:28.291162 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.280390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43129 (local address 127.31.210.195:58349)
0808 00:32:28.280585 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.280630 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.281149 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.281467 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.281475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.281801 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:28.282002 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.282007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.283053 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.283054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.284004 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.284007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.284439 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.284440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.284615 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.285173 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:28.285182 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:28.287075 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:28.289146 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.289150 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.289153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.289502 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.289762 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.289763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.289763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.289895 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.290422 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:28.290429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:28.290615 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.290750 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.290921 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":119,"threads_started":1}
I20260808 00:32:28.291676 3307 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43129
I20260808 00:32:28.291782 3307 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:28.291992 3307 heartbeater.cc:511] Master 127.31.210.254:43129 requested a full tablet report, sending...
I20260808 00:32:28.292433 2598 ts_manager.cc:194] Registered new tserver with Master: 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:28.292989 2598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:58349
I20260808 00:32:28.298650 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.292842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43129 (local address 127.31.210.195:54575)
0808 00:32:28.293063 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.293074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.293145 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.293277 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.293277 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.293465 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:28.293516 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.293518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.294183 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.294184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.294707 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.294708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.295131 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.295132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.295208 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.295708 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:28.295714 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:28.296093 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:28.297573 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.297574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.297575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.297751 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.297970 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.297970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.297971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.298037 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.298313 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:28.298314 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:28.298360 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.298439 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.298549 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":147,"threads_started":1}
I20260808 00:32:28.298650 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.292919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:54575 (local address 127.31.210.254:43129)
0808 00:32:28.292959 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.292961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.293130 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:32:28.293204 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.293206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.293216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.293272 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:28.293613 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.294084 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.294808 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.295016 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.295101 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.296182 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:28.296184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:28.296184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:28.296192 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:28.297470 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.297829 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.297830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.297830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.297892 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.298116 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.298117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.298117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.298235 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:28.298308 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.298477 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.298549 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:32:28.301909 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:32:28 dist-test-slave-44d2 krb5kdc[2540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:32:28 dist-test-slave-44d2 krb5kdc[2540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:28.328445 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.320569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.31.210.254:43129)
0808 00:32:28.320640 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.320642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.320651 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:28.320679 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.320680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.320691 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.320767 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:28.321436 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.321985 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.322834 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.323052 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.323144 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.325817 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:28.325818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:28.325819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:28.325826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:28.327140 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.327544 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.327546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.327546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.327603 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.327829 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.327830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.327830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.327959 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:28.328077 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.328231 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.328326 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:32:28.330129 2598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:28.331771 2598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:28.339303 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.334365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:40661 (local address 127.0.0.1:53574)
0808 00:32:28.334824 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.334834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.334916 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.335202 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.335203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.335205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.335251 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.335253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.336614 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.336615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.338093 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.338097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.338989 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.338990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.339045 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.339049 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.339109 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.339189 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":9,"thread_start_us":166,"threads_started":1}
I20260808 00:32:28.339282 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.334615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53574 (local address 127.31.210.195:40661)
0808 00:32:28.334829 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.334835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.334886 (+ 51us) server_negotiation.cc:407] Connection header received
0808 00:32:28.335030 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.335033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.335052 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.335111 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.335380 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.336431 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.338186 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.338900 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.338971 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.339019 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.339132 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.339176 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":102,"threads_started":1}
I20260808 00:32:28.339696 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.334842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.31.210.193:38099)
0808 00:32:28.335040 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.335046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.335056 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:28.335366 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.335369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.335387 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.335451 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.335700 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.336853 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.338046 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.338810 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.338901 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.338971 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.339547 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.339592 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":81,"threads_started":1}
I20260808 00:32:28.339304 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.334373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38099 (local address 127.0.0.1:37792)
0808 00:32:28.334860 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.334873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.335208 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.335559 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.335560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.335562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.335607 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.335608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.336931 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.336932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.337952 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.337955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.338903 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.338904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.339003 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.339021 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.339109 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.339187 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":32,"thread_start_us":146,"threads_started":1}
I20260808 00:32:28.340431 3332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.334362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:34791 (local address 127.0.0.1:47072)
0808 00:32:28.334591 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.334637 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.334713 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.335115 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.335118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.335120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.335198 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.335200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.336580 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.336581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.338097 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.338099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.340164 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.340165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.340244 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.340248 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.340306 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.340351 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":81,"thread_start_us":65,"threads_started":1}
I20260808 00:32:28.340447 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.334645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.31.210.194:34791)
0808 00:32:28.334919 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.334923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.334935 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:28.334981 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.334983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.335002 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.335096 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.335295 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.336429 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.339057 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.340078 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.340165 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.340219 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.340320 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.340356 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":117,"threads_started":1}
I20260808 00:32:28.340642 2804 tablet_service.cc:1467] Processing CreateTablet for tablet c2855cfcf4194a0c977f170ead790f9a (DEFAULT_TABLE table=test-table [id=b6fce9d6b1094b13a806caf88433d870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:28.341074 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2855cfcf4194a0c977f170ead790f9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.341172 3004 tablet_service.cc:1467] Processing CreateTablet for tablet c2855cfcf4194a0c977f170ead790f9a (DEFAULT_TABLE table=test-table [id=b6fce9d6b1094b13a806caf88433d870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:28.341538 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2855cfcf4194a0c977f170ead790f9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.344529 3341 tablet_bootstrap.cc:492] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Bootstrap starting.
I20260808 00:32:28.344776 3342 tablet_bootstrap.cc:492] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Bootstrap starting.
I20260808 00:32:28.345116 3209 tablet_service.cc:1467] Processing CreateTablet for tablet c2855cfcf4194a0c977f170ead790f9a (DEFAULT_TABLE table=test-table [id=b6fce9d6b1094b13a806caf88433d870]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:28.345317 3341 tablet_bootstrap.cc:654] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.345438 3342 tablet_bootstrap.cc:654] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.345477 3209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2855cfcf4194a0c977f170ead790f9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.345733 3341 log.cc:824] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:28.345809 3342 log.cc:824] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:28.346693 3341 tablet_bootstrap.cc:492] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: No bootstrap required, opened a new log
I20260808 00:32:28.346773 3341 ts_tablet_manager.cc:1397] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:32:28.346855 3342 tablet_bootstrap.cc:492] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: No bootstrap required, opened a new log
I20260808 00:32:28.346918 3342 ts_tablet_manager.cc:1397] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:28.348433 3342 raft_consensus.cc:348] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.348654 3342 raft_consensus.cc:374] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.348687 3342 raft_consensus.cc:729] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a117c33be64331b0eb35e9d6e309b5, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.348876 3349 tablet_bootstrap.cc:492] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Bootstrap starting.
I20260808 00:32:28.348878 3341 raft_consensus.cc:348] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.349021 3341 raft_consensus.cc:374] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.349057 3341 raft_consensus.cc:729] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.348996 3342 consensus_queue.cc:260] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.349169 3341 consensus_queue.cc:260] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.349320 3342 ts_tablet_manager.cc:1428] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:32:28.349341 2901 heartbeater.cc:503] Master 127.31.210.254:43129 was elected leader, sending a full tablet report...
I20260808 00:32:28.349563 3341 ts_tablet_manager.cc:1428] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:28.349670 3096 heartbeater.cc:503] Master 127.31.210.254:43129 was elected leader, sending a full tablet report...
I20260808 00:32:28.350656 3349 tablet_bootstrap.cc:654] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.351156 3349 log.cc:824] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:28.351907 3349 tablet_bootstrap.cc:492] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: No bootstrap required, opened a new log
I20260808 00:32:28.351969 3349 ts_tablet_manager.cc:1397] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:32:28.353557 3349 raft_consensus.cc:348] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.353693 3349 raft_consensus.cc:374] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.353716 3349 raft_consensus.cc:729] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a836c8948e49e2a1033839ae41f048, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.353801 3349 consensus_queue.cc:260] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.354015 3349 ts_tablet_manager.cc:1428] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:32:28.354067 3307 heartbeater.cc:503] Master 127.31.210.254:43129 was elected leader, sending a full tablet report...
W20260808 00:32:28.359941 3098 tablet.cc:2369] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:32:28.447768 2904 tablet.cc:2369] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:28.498421 3353 raft_consensus.cc:482] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:28.498558 3353 raft_consensus.cc:504] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.498932 3353 leader_election.cc:288] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791), 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099)
I20260808 00:32:28.502848 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.499226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:47435 (local address 127.31.210.194:34791)
0808 00:32:28.499276 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.499278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.499308 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:32:28.499342 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.499343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.499353 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.499408 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.499609 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.500363 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.501901 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.502475 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.502543 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.502553 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.502731 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.502771 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:32:28.502848 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.499279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:47303 (local address 127.31.210.193:38099)
0808 00:32:28.499322 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.499324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.499483 (+ 159us) server_negotiation.cc:407] Connection header received
0808 00:32:28.499561 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.499562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.499589 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.499648 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.499769 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.500455 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.501901 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.502457 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.502543 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.502553 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.502731 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.502771 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:32:28.502880 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.499123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:34791 (local address 127.31.210.195:47435)
0808 00:32:28.499186 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.499199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.499255 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.499436 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.499437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.499439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.499530 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.499533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.500447 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.500448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.501806 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.501807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.502561 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.502563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.502662 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.502665 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.502725 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.502784 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":117}
I20260808 00:32:28.502895 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.499223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38099 (local address 127.31.210.195:47303)
0808 00:32:28.499404 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.499414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.499467 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.499660 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.499661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.499662 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.499715 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.499716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.500570 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.500570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.501801 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.501806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.502561 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.502563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.502650 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.502655 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.502725 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.502784 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":8,"thread_start_us":110,"threads_started":1}
I20260808 00:32:28.503331 3034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2855cfcf4194a0c977f170ead790f9a" candidate_uuid: "46a836c8948e49e2a1033839ae41f048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016bfba080a042e2b38f6db8d0ac1616" is_pre_election: true
I20260808 00:32:28.503533 3034 raft_consensus.cc:2454] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a836c8948e49e2a1033839ae41f048 in term 0.
I20260808 00:32:28.503506 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2855cfcf4194a0c977f170ead790f9a" candidate_uuid: "46a836c8948e49e2a1033839ae41f048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a117c33be64331b0eb35e9d6e309b5" is_pre_election: true
I20260808 00:32:28.503639 2837 raft_consensus.cc:2454] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46a836c8948e49e2a1033839ae41f048 in term 0.
I20260808 00:32:28.503779 3133 leader_election.cc:302] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 46a836c8948e49e2a1033839ae41f048; no voters:
I20260808 00:32:28.503979 3353 raft_consensus.cc:2781] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:28.504033 3353 raft_consensus.cc:482] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:28.504050 3353 raft_consensus.cc:3037] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.504895 3353 raft_consensus.cc:504] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.505061 3353 leader_election.cc:288] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [CANDIDATE]: Term 1 election: Requested vote from peers 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791), 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099)
I20260808 00:32:28.505300 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2855cfcf4194a0c977f170ead790f9a" candidate_uuid: "46a836c8948e49e2a1033839ae41f048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a117c33be64331b0eb35e9d6e309b5"
I20260808 00:32:28.505384 2837 raft_consensus.cc:3037] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.506163 2837 raft_consensus.cc:2454] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a836c8948e49e2a1033839ae41f048 in term 1.
I20260808 00:32:28.506299 3034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2855cfcf4194a0c977f170ead790f9a" candidate_uuid: "46a836c8948e49e2a1033839ae41f048" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016bfba080a042e2b38f6db8d0ac1616"
I20260808 00:32:28.506386 3034 raft_consensus.cc:3037] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.506788 3130 leader_election.cc:302] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42a117c33be64331b0eb35e9d6e309b5, 46a836c8948e49e2a1033839ae41f048; no voters:
I20260808 00:32:28.506891 3353 raft_consensus.cc:2781] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:28.507074 3353 raft_consensus.cc:686] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 1 LEADER]: Becoming Leader. State: Replica: 46a836c8948e49e2a1033839ae41f048, State: Running, Role: LEADER
I20260808 00:32:28.507189 3353 consensus_queue.cc:237] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } }
I20260808 00:32:28.507320 3034 raft_consensus.cc:2454] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46a836c8948e49e2a1033839ae41f048 in term 1.
I20260808 00:32:28.508085 2598 catalog_manager.cc:5495] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46a836c8948e49e2a1033839ae41f048 (127.31.210.195). New cstate: current_term: 1 leader_uuid: "46a836c8948e49e2a1033839ae41f048" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } health_report { overall_health: UNKNOWN } } }
W20260808 00:32:28.539486 3308 tablet.cc:2369] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 00:32:28 dist-test-slave-44d2 krb5kdc[2540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149147, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:28.557876 3364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.549602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43129 (local address 127.0.0.1:44336)
0808 00:32:28.549785 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.549795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.549954 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.550162 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.550162 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.550622 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:28.550683 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.550687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.551644 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.551645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.552210 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.552212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.552673 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.552674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.552730 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.553312 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:28.553317 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:28.554906 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:28.556614 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.556623 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.556626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.556847 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.557090 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:28.557092 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:28.557092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:28.557160 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:28.557446 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:28.557455 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:28.557507 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.557597 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.557728 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260808 00:32:28.557971 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.549641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.31.210.254:43129)
0808 00:32:28.549709 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.549712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.549928 (+ 216us) server_negotiation.cc:407] Connection header received
0808 00:32:28.550047 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.550048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.550061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.550152 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:28.550804 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.551522 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.552308 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.552558 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.552671 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.555032 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:28.555034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:28.555034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:28.555042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:28.556431 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.556949 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.556950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.556951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.557016 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:28.557229 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:28.557230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:28.557230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:28.557353 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:28.557444 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.557639 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.557781 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":34}
I20260808 00:32:28.560277 2598 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:44336:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:32:28.560534 2598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:28.562379 3004 tablet_service.cc:1467] Processing CreateTablet for tablet 289f42548fcc45689f581d32912af571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9ce51ee5b4a140268406c061694fae8c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:28.562422 3209 tablet_service.cc:1467] Processing CreateTablet for tablet 289f42548fcc45689f581d32912af571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9ce51ee5b4a140268406c061694fae8c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:28.562590 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289f42548fcc45689f581d32912af571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.562525 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 289f42548fcc45689f581d32912af571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9ce51ee5b4a140268406c061694fae8c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:28.562590 3209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289f42548fcc45689f581d32912af571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.562711 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289f42548fcc45689f581d32912af571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.563850 3342 tablet_bootstrap.cc:492] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: Bootstrap starting.
I20260808 00:32:28.564199 3341 tablet_bootstrap.cc:492] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: Bootstrap starting.
I20260808 00:32:28.564199 3349 tablet_bootstrap.cc:492] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: Bootstrap starting.
I20260808 00:32:28.564358 3342 tablet_bootstrap.cc:654] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.564726 3349 tablet_bootstrap.cc:654] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.565024 3341 tablet_bootstrap.cc:654] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.565056 3342 tablet_bootstrap.cc:492] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: No bootstrap required, opened a new log
I20260808 00:32:28.565112 3342 ts_tablet_manager.cc:1397] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:28.565295 3342 raft_consensus.cc:348] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.565382 3342 raft_consensus.cc:374] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.565412 3342 raft_consensus.cc:729] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a117c33be64331b0eb35e9d6e309b5, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.565457 3342 consensus_queue.cc:260] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.565544 3349 tablet_bootstrap.cc:492] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: No bootstrap required, opened a new log
I20260808 00:32:28.565578 3342 ts_tablet_manager.cc:1428] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.565598 3349 ts_tablet_manager.cc:1397] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:28.565656 3350 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.565729 3350 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:28.565754 3341 tablet_bootstrap.cc:492] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: No bootstrap required, opened a new log
I20260808 00:32:28.565799 3341 ts_tablet_manager.cc:1397] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:32:28.565774 3349 raft_consensus.cc:348] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.565874 3349 raft_consensus.cc:374] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.565907 3349 raft_consensus.cc:729] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a836c8948e49e2a1033839ae41f048, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.565950 3349 consensus_queue.cc:260] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.565958 3341 raft_consensus.cc:348] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.566027 3341 raft_consensus.cc:374] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.566051 3341 raft_consensus.cc:729] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.566051 3349 ts_tablet_manager.cc:1428] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:28.566099 3341 consensus_queue.cc:260] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.566099 3353 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.566193 3353 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:28.566213 3341 ts_tablet_manager.cc:1428] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.566268 3351 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.566385 3351 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:28.625065 3350 raft_consensus.cc:482] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:28.625198 3350 raft_consensus.cc:504] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.625617 3350 leader_election.cc:288] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:28.628867 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.625848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:34491 (local address 127.31.210.194:34791)
0808 00:32:28.625896 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.625898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.626107 (+ 209us) server_negotiation.cc:407] Connection header received
0808 00:32:28.626212 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.626214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.626225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.626285 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.626505 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.627028 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.628050 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.628432 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.628564 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.628573 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.628737 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.628777 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:32:28.628985 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.626046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:46117 (local address 127.31.210.195:40661)
0808 00:32:28.626122 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.626124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.626137 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:32:28.626249 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.626250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.626280 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.626335 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.626530 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.627034 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.628119 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.628599 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.628679 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.628689 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.628871 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.628928 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:32:28.629027 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.625922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:40661 (local address 127.31.210.193:46117)
0808 00:32:28.626074 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.626085 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.626146 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.626360 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.626361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.626363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.626435 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.626452 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.627139 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.627140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.627954 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.627956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.628685 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.628686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.628784 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.628789 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.628849 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.628908 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":55,"thread_start_us":58,"threads_started":1}
I20260808 00:32:28.629148 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.625804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:34791 (local address 127.31.210.193:34491)
0808 00:32:28.626028 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.626041 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.626132 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.626301 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.626302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.626304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.626392 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.626394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.627121 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.627122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.627958 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.627960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.628577 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.628579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.628658 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.628664 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.628722 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.628767 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":70,"threads_started":1}
I20260808 00:32:28.629415 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289f42548fcc45689f581d32912af571" candidate_uuid: "42a117c33be64331b0eb35e9d6e309b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048" is_pre_election: true
I20260808 00:32:28.629552 3234 raft_consensus.cc:2454] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42a117c33be64331b0eb35e9d6e309b5 in term 0.
I20260808 00:32:28.629565 3034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289f42548fcc45689f581d32912af571" candidate_uuid: "42a117c33be64331b0eb35e9d6e309b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016bfba080a042e2b38f6db8d0ac1616" is_pre_election: true
I20260808 00:32:28.629670 3034 raft_consensus.cc:2454] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42a117c33be64331b0eb35e9d6e309b5 in term 0.
I20260808 00:32:28.629791 2733 leader_election.cc:302] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42a117c33be64331b0eb35e9d6e309b5, 46a836c8948e49e2a1033839ae41f048; no voters:
I20260808 00:32:28.629969 3350 raft_consensus.cc:2781] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:28.630015 3350 raft_consensus.cc:482] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:28.630038 3350 raft_consensus.cc:3037] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.630725 3350 raft_consensus.cc:504] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.630879 3350 leader_election.cc:288] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [CANDIDATE]: Term 1 election: Requested vote from peers 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:28.631057 3034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289f42548fcc45689f581d32912af571" candidate_uuid: "42a117c33be64331b0eb35e9d6e309b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016bfba080a042e2b38f6db8d0ac1616"
I20260808 00:32:28.631050 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289f42548fcc45689f581d32912af571" candidate_uuid: "42a117c33be64331b0eb35e9d6e309b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048"
I20260808 00:32:28.631138 3234 raft_consensus.cc:3037] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.631134 3034 raft_consensus.cc:3037] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.631781 3034 raft_consensus.cc:2454] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42a117c33be64331b0eb35e9d6e309b5 in term 1.
I20260808 00:32:28.631796 3234 raft_consensus.cc:2454] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42a117c33be64331b0eb35e9d6e309b5 in term 1.
I20260808 00:32:28.632025 2736 leader_election.cc:302] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 42a117c33be64331b0eb35e9d6e309b5; no voters:
I20260808 00:32:28.632124 3350 raft_consensus.cc:2781] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:28.632252 3350 raft_consensus.cc:686] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 LEADER]: Becoming Leader. State: Replica: 42a117c33be64331b0eb35e9d6e309b5, State: Running, Role: LEADER
I20260808 00:32:28.632355 3350 consensus_queue.cc:237] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.632881 3372 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: TxnStatusTablet state changed. Reason: New leader 42a117c33be64331b0eb35e9d6e309b5. Latest consensus state: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.633034 3372 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:28.633252 3374 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:32:28.633247 2597 catalog_manager.cc:5495] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193). New cstate: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:28.633383 3374 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:32:28.644622 3364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.641477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38099 (local address 127.0.0.1:37804)
0808 00:32:28.641587 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.641602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.641665 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.641839 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.641839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.641840 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:28.641955 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.641958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.642860 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.642861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.643562 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.643565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.644038 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.644039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.644095 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.644112 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.644407 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.644417 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.644459 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.644564 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 00:32:28.644621 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.641590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.31.210.193:38099)
0808 00:32:28.641665 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.641668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.641680 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:28.641737 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.641738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.641758 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.641838 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:28.642027 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.642770 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.643648 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.643953 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.644027 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.644204 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.644331 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.644412 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.644471 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.644539 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:32:28.646842 3234 raft_consensus.cc:1261] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Refusing update from remote peer 42a117c33be64331b0eb35e9d6e309b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:28.646890 3034 raft_consensus.cc:1261] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Refusing update from remote peer 42a117c33be64331b0eb35e9d6e309b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:28.647221 3372 consensus_queue.cc:1035] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:28.647387 3350 consensus_queue.cc:1035] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:28.648870 3351 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: TxnStatusTablet state changed. Reason: New leader 42a117c33be64331b0eb35e9d6e309b5. Latest consensus state: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.648979 3351 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:28.650701 3353 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: TxnStatusTablet state changed. Reason: New leader 42a117c33be64331b0eb35e9d6e309b5. Latest consensus state: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.650795 3353 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:28.650861 3351 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.650965 3351 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 016bfba080a042e2b38f6db8d0ac1616: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:28.651003 3350 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.651079 3350 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:28.651348 3377 mvcc.cc:205] Tried to move back new op lower bound from 7316066912854130688 to 7316066912798760960. Current Snapshot: MvccSnapshot[applied={T|T < 7316066912854130688}]
I20260808 00:32:28.652007 3378 mvcc.cc:205] Tried to move back new op lower bound from 7316066912854130688 to 7316066912798760960. Current Snapshot: MvccSnapshot[applied={T|T < 7316066912854130688}]
I20260808 00:32:28.657212 3350 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.657318 3350 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 42a117c33be64331b0eb35e9d6e309b5: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:28.657599 3353 tablet_replica.cc:439] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42a117c33be64331b0eb35e9d6e309b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } } }
I20260808 00:32:28.657704 3353 tablet_replica.cc:442] T 289f42548fcc45689f581d32912af571 P 46a836c8948e49e2a1033839ae41f048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:28.658017 3383 mvcc.cc:205] Tried to move back new op lower bound from 7316066912854130688 to 7316066912798760960. Current Snapshot: MvccSnapshot[applied={T|T < 7316066912854130688}]
I20260808 00:32:28.659425 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.656426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.31.210.195:40661)
0808 00:32:28.656469 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.656471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.656505 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:32:28.656573 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.656574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.656599 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.656682 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:28.656841 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.657572 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.658596 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.658907 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.658982 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.659092 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.659215 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.659290 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.659313 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.659335 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:32:28.664597 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.662107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:45519 (local address 127.31.210.193:38099)
0808 00:32:28.662159 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.662161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.662227 (+ 66us) server_negotiation.cc:407] Connection header received
0808 00:32:28.662248 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.662249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.662259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.662326 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:28.662471 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.662957 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.663790 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.664031 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.664110 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.664308 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.664342 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.664408 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.664502 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.664520 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:32:28.664597 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.662057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38099 (local address 127.31.210.195:45519)
0808 00:32:28.662126 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.662138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.662189 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.662331 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.662332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.662333 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:28.662389 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.662390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.663051 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.663053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.663699 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.663701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.664109 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.664110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.664175 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.664196 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.664424 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.664426 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.664492 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.664533 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 00:32:28.667117 3034 raft_consensus.cc:1261] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Refusing update from remote peer 46a836c8948e49e2a1033839ae41f048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:28.667240 2837 raft_consensus.cc:1261] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Refusing update from remote peer 46a836c8948e49e2a1033839ae41f048: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:28.667389 3355 consensus_queue.cc:1035] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:28.667701 3355 consensus_queue.cc:1035] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:28.676386 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.673286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:40661 (local address 127.31.210.193:43931)
0808 00:32:28.673441 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.673453 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.673515 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.673654 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.673655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.673656 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:28.673716 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.673717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.674572 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.674573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.675500 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.675503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.675984 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.675985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.676042 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.676058 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.676245 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.676247 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.676281 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.676315 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260808 00:32:28.676385 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.673460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:43931 (local address 127.31.210.195:40661)
0808 00:32:28.673497 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.673499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.673509 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:28.673579 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.673580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.673590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.673653 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:28.673752 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.674496 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.675580 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.675879 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.675951 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.676148 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.676183 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.676239 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.676289 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.676310 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:32:28.686040 2598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:28.686157 2598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:28.687014 2598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Sending DeleteTablet for 3 replicas of tablet c2855cfcf4194a0c977f170ead790f9a
I20260808 00:32:28.687373 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet c2855cfcf4194a0c977f170ead790f9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:28 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:37792
I20260808 00:32:28.687655 3400 tablet_replica.cc:330] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: stopping tablet replica
I20260808 00:32:28.687868 3400 raft_consensus.cc:2229] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:28.687870 3004 tablet_service.cc:1514] Processing DeleteTablet for tablet c2855cfcf4194a0c977f170ead790f9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:28 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:47072
I20260808 00:32:28.687989 3400 raft_consensus.cc:2258] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:28.688423 3209 tablet_service.cc:1514] Processing DeleteTablet for tablet c2855cfcf4194a0c977f170ead790f9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:28 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:53574
I20260808 00:32:28.688472 3400 ts_tablet_manager.cc:1905] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:28.688719 2598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:28.688855 2598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:28.690476 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 00ffcc6506aa45b7a2b158781c0ea728 (DEFAULT_TABLE table=test-table [id=d160221b66694873a4c3926980b47165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:28.690593 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ffcc6506aa45b7a2b158781c0ea728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.691016 3004 tablet_service.cc:1467] Processing CreateTablet for tablet 00ffcc6506aa45b7a2b158781c0ea728 (DEFAULT_TABLE table=test-table [id=d160221b66694873a4c3926980b47165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:28.691135 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ffcc6506aa45b7a2b158781c0ea728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.692225 3342 tablet_bootstrap.cc:492] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: Bootstrap starting.
I20260808 00:32:28.692658 3341 tablet_bootstrap.cc:492] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: Bootstrap starting.
I20260808 00:32:28.692708 3342 tablet_bootstrap.cc:654] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.693041 3341 tablet_bootstrap.cc:654] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.693051 3400 ts_tablet_manager.cc:1918] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:28.693118 3400 log.cc:1196] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/c2855cfcf4194a0c977f170ead790f9a
I20260808 00:32:28.693403 3400 ts_tablet_manager.cc:1939] T c2855cfcf4194a0c977f170ead790f9a P 42a117c33be64331b0eb35e9d6e309b5: Deleting consensus metadata
I20260808 00:32:28.693826 2579 catalog_manager.cc:4841] TS 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099): tablet c2855cfcf4194a0c977f170ead790f9a (table test-table [id=b6fce9d6b1094b13a806caf88433d870]) successfully deleted
I20260808 00:32:28.693917 3209 tablet_service.cc:1467] Processing CreateTablet for tablet 00ffcc6506aa45b7a2b158781c0ea728 (DEFAULT_TABLE table=test-table [id=d160221b66694873a4c3926980b47165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:28.694036 3209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ffcc6506aa45b7a2b158781c0ea728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:28.695305 3349 tablet_bootstrap.cc:492] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: Bootstrap starting.
I20260808 00:32:28.695742 3349 tablet_bootstrap.cc:654] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:28.696666 3349 tablet_bootstrap.cc:492] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: No bootstrap required, opened a new log
I20260808 00:32:28.696719 3349 ts_tablet_manager.cc:1397] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:28.696938 3349 raft_consensus.cc:348] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.697013 3349 raft_consensus.cc:374] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.697036 3349 raft_consensus.cc:729] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a836c8948e49e2a1033839ae41f048, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.697078 3349 consensus_queue.cc:260] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.697116 3341 tablet_bootstrap.cc:492] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: No bootstrap required, opened a new log
I20260808 00:32:28.697162 3341 ts_tablet_manager.cc:1397] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 00:32:28.697167 3349 ts_tablet_manager.cc:1428] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.697305 3341 raft_consensus.cc:348] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.697381 3341 raft_consensus.cc:374] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.697400 3341 raft_consensus.cc:729] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.697441 3341 consensus_queue.cc:260] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.697541 3341 ts_tablet_manager.cc:1428] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.697990 3402 tablet_replica.cc:330] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: stopping tablet replica
I20260808 00:32:28.698103 3402 raft_consensus.cc:2229] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:28.698203 3402 raft_consensus.cc:2258] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:28.698451 3401 tablet_replica.cc:330] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: stopping tablet replica
I20260808 00:32:28.698573 3401 raft_consensus.cc:2229] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:28.698632 3401 raft_consensus.cc:2258] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:28.699285 3402 ts_tablet_manager.cc:1905] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:28.699378 3401 ts_tablet_manager.cc:1905] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:28.701270 3402 ts_tablet_manager.cc:1918] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:28.701397 3402 log.cc:1196] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/c2855cfcf4194a0c977f170ead790f9a
I20260808 00:32:28.701603 3401 ts_tablet_manager.cc:1918] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:28.701731 3401 log.cc:1196] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/c2855cfcf4194a0c977f170ead790f9a
I20260808 00:32:28.701731 3402 ts_tablet_manager.cc:1939] T c2855cfcf4194a0c977f170ead790f9a P 46a836c8948e49e2a1033839ae41f048: Deleting consensus metadata
I20260808 00:32:28.702061 3401 ts_tablet_manager.cc:1939] T c2855cfcf4194a0c977f170ead790f9a P 016bfba080a042e2b38f6db8d0ac1616: Deleting consensus metadata
I20260808 00:32:28.702144 2582 catalog_manager.cc:4841] TS 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661): tablet c2855cfcf4194a0c977f170ead790f9a (table test-table [id=b6fce9d6b1094b13a806caf88433d870]) successfully deleted
I20260808 00:32:28.702452 3342 tablet_bootstrap.cc:492] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: No bootstrap required, opened a new log
I20260808 00:32:28.702517 3342 ts_tablet_manager.cc:1397] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260808 00:32:28.702661 3342 raft_consensus.cc:348] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.702730 3342 raft_consensus.cc:374] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:28.702749 3342 raft_consensus.cc:729] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a117c33be64331b0eb35e9d6e309b5, State: Initialized, Role: FOLLOWER
I20260808 00:32:28.702788 3342 consensus_queue.cc:260] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.702793 2584 catalog_manager.cc:4841] TS 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791): tablet c2855cfcf4194a0c977f170ead790f9a (table test-table [id=b6fce9d6b1094b13a806caf88433d870]) successfully deleted
I20260808 00:32:28.702864 3342 ts_tablet_manager.cc:1428] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:28.728065 3351 raft_consensus.cc:482] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:28.728166 3351 raft_consensus.cc:504] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.728523 3351 leader_election.cc:288] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:28.733409 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.729269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:57261 (local address 127.31.210.195:40661)
0808 00:32:28.729312 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.729315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.729882 (+ 567us) server_negotiation.cc:407] Connection header received
0808 00:32:28.729995 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.729996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.730006 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.730034 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.730241 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.730964 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.732371 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.732985 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.733035 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.733044 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.733298 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.733336 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:32:28.733559 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.729595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:34069 (local address 127.31.210.193:38099)
0808 00:32:28.729640 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.729642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.729650 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:28.729678 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.729679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.729689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.729739 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:28.729977 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.730703 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.732192 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.732803 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.732859 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.732869 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.733436 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.733490 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:32:28.733924 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.728807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38099 (local address 127.31.210.194:34069)
0808 00:32:28.729362 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.729374 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.729430 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.729796 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.729798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.729801 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.729885 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.729887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.730811 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.730812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.732080 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.732083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.733262 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.733263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.733344 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.733348 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.733751 (+ 403us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.733825 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":30,"thread_start_us":66,"threads_started":1}
I20260808 00:32:28.733937 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.729124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:40661 (local address 127.31.210.194:57261)
0808 00:32:28.729806 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.729818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.729913 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.730098 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.730098 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.730100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:28.730169 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.730170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.731069 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.731070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.732257 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.732261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.733118 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.733119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.733212 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.733217 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.733799 (+ 582us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.733846 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":46,"threads_started":1}
I20260808 00:32:28.734277 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ffcc6506aa45b7a2b158781c0ea728" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a117c33be64331b0eb35e9d6e309b5" is_pre_election: true
I20260808 00:32:28.734375 2837 raft_consensus.cc:2454] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 0.
I20260808 00:32:28.734464 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ffcc6506aa45b7a2b158781c0ea728" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048" is_pre_election: true
I20260808 00:32:28.734570 3234 raft_consensus.cc:2454] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 0.
I20260808 00:32:28.734606 2939 leader_election.cc:302] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 42a117c33be64331b0eb35e9d6e309b5; no voters:
I20260808 00:32:28.734777 3351 raft_consensus.cc:2781] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:28.734822 3351 raft_consensus.cc:482] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:28.734843 3351 raft_consensus.cc:3037] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.735773 3351 raft_consensus.cc:504] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.735922 3351 leader_election.cc:288] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 election: Requested vote from peers 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:28.736136 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ffcc6506aa45b7a2b158781c0ea728" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048"
I20260808 00:32:28.736205 3234 raft_consensus.cc:3037] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.736176 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ffcc6506aa45b7a2b158781c0ea728" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a117c33be64331b0eb35e9d6e309b5"
I20260808 00:32:28.736269 2837 raft_consensus.cc:3037] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:28.737022 3234 raft_consensus.cc:2454] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 1.
I20260808 00:32:28.737170 2837 raft_consensus.cc:2454] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 1.
I20260808 00:32:28.737213 2940 leader_election.cc:302] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 46a836c8948e49e2a1033839ae41f048; no voters:
I20260808 00:32:28.737403 3351 raft_consensus.cc:2781] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:28.737569 3351 raft_consensus.cc:686] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 LEADER]: Becoming Leader. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Running, Role: LEADER
I20260808 00:32:28.737641 3351 consensus_queue.cc:237] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:28.738349 2597 catalog_manager.cc:5495] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194). New cstate: current_term: 1 leader_uuid: "016bfba080a042e2b38f6db8d0ac1616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:28.750582 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.747158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47084 (local address 127.31.210.194:34791)
0808 00:32:28.747198 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.747200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.747228 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:32:28.747308 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.747309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.747335 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.747420 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:28.747569 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.748287 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.749455 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.749778 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.749856 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.750045 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.750181 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.750283 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.750452 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.750485 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:32:28.757012 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.753965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:46289 (local address 127.31.210.193:38099)
0808 00:32:28.754006 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.754008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.754016 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:28.754075 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.754075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.754088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.754147 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:28.754314 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.755047 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.756116 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.756422 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.756543 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.756741 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.756778 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.756839 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.756909 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.756933 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:32:28.757046 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.753776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38099 (local address 127.31.210.194:46289)
0808 00:32:28.753951 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.753963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.754019 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.754161 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.754162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.754164 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:28.754236 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.754238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.755141 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.755142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.756016 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.756019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.756543 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.756545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.756616 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.756633 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.756854 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.756857 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.756906 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.756953 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260808 00:32:28.759737 2837 raft_consensus.cc:1261] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Refusing update from remote peer 016bfba080a042e2b38f6db8d0ac1616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:28.759743 3234 raft_consensus.cc:1261] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Refusing update from remote peer 016bfba080a042e2b38f6db8d0ac1616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:28.760037 3351 consensus_queue.cc:1035] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:28.760192 3410 consensus_queue.cc:1035] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:28.770457 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.767276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:47381 (local address 127.31.210.194:34791)
0808 00:32:28.767345 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:28.767347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:28.767353 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:32:28.767373 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:28.767374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:28.767383 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:28.767423 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:28.767602 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.768289 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.769628 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.769902 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.769978 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.770167 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.770203 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.770256 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:28.770361 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:32:28.770382 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:32:28.770483 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:28.766966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:34791 (local address 127.31.210.193:47381)
0808 00:32:28.767076 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:28.767088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:28.767128 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:28.767468 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:28.767469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:28.767471 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:28.767526 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.767528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.768388 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.768390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.769516 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:28.769520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:28.770018 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:28.770019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:28.770077 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:28.770085 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:28.770279 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:28.770283 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:28.770341 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:32:28.770391 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
W20260808 00:32:28.771111 3378 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 00:32:30.299151 2598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:30.299266 2598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:30.300159 2598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Sending DeleteTablet for 3 replicas of tablet 00ffcc6506aa45b7a2b158781c0ea728
I20260808 00:32:30.300465 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ffcc6506aa45b7a2b158781c0ea728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:37792
I20260808 00:32:30.300647 3004 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ffcc6506aa45b7a2b158781c0ea728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:47072
I20260808 00:32:30.300978 3461 tablet_replica.cc:330] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: stopping tablet replica
I20260808 00:32:30.301070 3461 raft_consensus.cc:2229] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:30.301126 3461 raft_consensus.cc:2258] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:30.301415 3461 ts_tablet_manager.cc:1905] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:30.302309 3462 tablet_replica.cc:330] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: stopping tablet replica
I20260808 00:32:30.302475 3462 raft_consensus.cc:2229] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:30.300585 3209 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ffcc6506aa45b7a2b158781c0ea728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:53574
I20260808 00:32:30.302673 3462 raft_consensus.cc:2258] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:30.302974 3462 ts_tablet_manager.cc:1905] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:30.304122 3464 tablet_replica.cc:330] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: stopping tablet replica
I20260808 00:32:30.304222 3464 raft_consensus.cc:2229] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:30.304276 3464 raft_consensus.cc:2258] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:30.304961 3464 ts_tablet_manager.cc:1905] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 00:32:30 dist-test-slave-44d2 krb5kdc[2540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149150, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260808 00:32:30.348526 3464 ts_tablet_manager.cc:1918] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:32:30.348531 3462 ts_tablet_manager.cc:1918] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:32:30.348637 3464 log.cc:1196] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/00ffcc6506aa45b7a2b158781c0ea728
I20260808 00:32:30.348644 3462 log.cc:1196] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/00ffcc6506aa45b7a2b158781c0ea728
I20260808 00:32:30.348779 3461 ts_tablet_manager.cc:1918] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:32:30.348873 3461 log.cc:1196] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/00ffcc6506aa45b7a2b158781c0ea728
I20260808 00:32:30.348927 3464 ts_tablet_manager.cc:1939] T 00ffcc6506aa45b7a2b158781c0ea728 P 46a836c8948e49e2a1033839ae41f048: Deleting consensus metadata
I20260808 00:32:30.348949 3462 ts_tablet_manager.cc:1939] T 00ffcc6506aa45b7a2b158781c0ea728 P 016bfba080a042e2b38f6db8d0ac1616: Deleting consensus metadata
I20260808 00:32:30.349156 3461 ts_tablet_manager.cc:1939] T 00ffcc6506aa45b7a2b158781c0ea728 P 42a117c33be64331b0eb35e9d6e309b5: Deleting consensus metadata
Aug 08 00:32:30 dist-test-slave-44d2 krb5kdc[2540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149150, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:30.359125 2582 catalog_manager.cc:4841] TS 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661): tablet 00ffcc6506aa45b7a2b158781c0ea728 (table test-table [id=d160221b66694873a4c3926980b47165]) successfully deleted
I20260808 00:32:30.359144 2579 catalog_manager.cc:4841] TS 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099): tablet 00ffcc6506aa45b7a2b158781c0ea728 (table test-table [id=d160221b66694873a4c3926980b47165]) successfully deleted
I20260808 00:32:30.359145 2584 catalog_manager.cc:4841] TS 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791): tablet 00ffcc6506aa45b7a2b158781c0ea728 (table test-table [id=d160221b66694873a4c3926980b47165]) successfully deleted
I20260808 00:32:30.361086 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:30.350928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.31.210.254:43129)
0808 00:32:30.351114 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:30.351117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:30.351214 (+ 97us) server_negotiation.cc:407] Connection header received
0808 00:32:30.351291 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:30.351293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:30.351308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:30.351385 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:30.352077 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:30.353138 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:30.354009 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:30.354649 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:30.354718 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:30.357435 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:30.357437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:30.357437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:30.357449 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:30.359676 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:30.360084 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:30.360086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:30.360086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:30.360161 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:30.360393 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:30.360394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:30.360394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:30.360594 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:30.360725 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:30.360867 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 00:32:30.360966 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260808 00:32:30.362761 2598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:30.362959 2598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:30.364728 3004 tablet_service.cc:1467] Processing CreateTablet for tablet feb03e3a7da04f1394a788dbcf844af2 (DEFAULT_TABLE table=test-table [id=a62a9d8edac8418c860b2f0c9604fa15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:30.364728 3209 tablet_service.cc:1467] Processing CreateTablet for tablet feb03e3a7da04f1394a788dbcf844af2 (DEFAULT_TABLE table=test-table [id=a62a9d8edac8418c860b2f0c9604fa15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:30.364738 2804 tablet_service.cc:1467] Processing CreateTablet for tablet feb03e3a7da04f1394a788dbcf844af2 (DEFAULT_TABLE table=test-table [id=a62a9d8edac8418c860b2f0c9604fa15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:30.364921 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb03e3a7da04f1394a788dbcf844af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:30.364892 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb03e3a7da04f1394a788dbcf844af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:30.364964 3209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb03e3a7da04f1394a788dbcf844af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:30.366509 3477 tablet_bootstrap.cc:492] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: Bootstrap starting.
I20260808 00:32:30.366662 3476 tablet_bootstrap.cc:492] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: Bootstrap starting.
I20260808 00:32:30.367003 3477 tablet_bootstrap.cc:654] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:30.367059 3476 tablet_bootstrap.cc:654] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:30.367405 3474 tablet_bootstrap.cc:492] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: Bootstrap starting.
I20260808 00:32:30.367750 3476 tablet_bootstrap.cc:492] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: No bootstrap required, opened a new log
I20260808 00:32:30.367862 3476 ts_tablet_manager.cc:1397] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:30.367801 3477 tablet_bootstrap.cc:492] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: No bootstrap required, opened a new log
I20260808 00:32:30.367910 3474 tablet_bootstrap.cc:654] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:30.368067 3477 ts_tablet_manager.cc:1397] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:30.368059 3476 raft_consensus.cc:348] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.368153 3476 raft_consensus.cc:374] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:30.368238 3476 raft_consensus.cc:729] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Initialized, Role: FOLLOWER
I20260808 00:32:30.368263 3477 raft_consensus.cc:348] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.368358 3477 raft_consensus.cc:374] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:30.368355 3476 consensus_queue.cc:260] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.368419 3477 raft_consensus.cc:729] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a117c33be64331b0eb35e9d6e309b5, State: Initialized, Role: FOLLOWER
I20260808 00:32:30.368549 3476 ts_tablet_manager.cc:1428] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:30.368610 3474 tablet_bootstrap.cc:492] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: No bootstrap required, opened a new log
I20260808 00:32:30.368654 3474 ts_tablet_manager.cc:1397] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:30.368791 3474 raft_consensus.cc:348] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.368858 3474 raft_consensus.cc:374] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:30.368873 3474 raft_consensus.cc:729] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a836c8948e49e2a1033839ae41f048, State: Initialized, Role: FOLLOWER
I20260808 00:32:30.368968 3474 consensus_queue.cc:260] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.368775 3477 consensus_queue.cc:260] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.369148 3474 ts_tablet_manager.cc:1428] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:30.369234 3477 ts_tablet_manager.cc:1428] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:30.524129 3444 raft_consensus.cc:482] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:30.524256 3444 raft_consensus.cc:504] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.524459 3444 leader_election.cc:288] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:30.524784 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb03e3a7da04f1394a788dbcf844af2" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048" is_pre_election: true
I20260808 00:32:30.524781 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb03e3a7da04f1394a788dbcf844af2" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a117c33be64331b0eb35e9d6e309b5" is_pre_election: true
I20260808 00:32:30.524883 3234 raft_consensus.cc:2454] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 0.
I20260808 00:32:30.524904 2837 raft_consensus.cc:2454] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 0.
I20260808 00:32:30.525084 2940 leader_election.cc:302] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 46a836c8948e49e2a1033839ae41f048; no voters:
I20260808 00:32:30.525197 3444 raft_consensus.cc:2781] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:30.525260 3444 raft_consensus.cc:482] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:30.525277 3444 raft_consensus.cc:3037] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:30.550949 3444 raft_consensus.cc:504] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.551314 3444 leader_election.cc:288] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 election: Requested vote from peers 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:30.551596 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb03e3a7da04f1394a788dbcf844af2" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048"
I20260808 00:32:30.551584 2837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb03e3a7da04f1394a788dbcf844af2" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a117c33be64331b0eb35e9d6e309b5"
I20260808 00:32:30.551718 3234 raft_consensus.cc:3037] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:30.551718 2837 raft_consensus.cc:3037] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:30.552692 3234 raft_consensus.cc:2454] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 1.
I20260808 00:32:30.552692 2837 raft_consensus.cc:2454] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 1.
I20260808 00:32:30.552927 2940 leader_election.cc:302] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 46a836c8948e49e2a1033839ae41f048; no voters:
I20260808 00:32:30.553121 3444 raft_consensus.cc:2781] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:30.553221 3444 raft_consensus.cc:686] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 LEADER]: Becoming Leader. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Running, Role: LEADER
I20260808 00:32:30.553288 3444 consensus_queue.cc:237] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.554054 2598 catalog_manager.cc:5495] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194). New cstate: current_term: 1 leader_uuid: "016bfba080a042e2b38f6db8d0ac1616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:30.600754 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:30.598001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.31.210.194:34791)
0808 00:32:30.598168 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:30.598170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:30.598177 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:32:30.598196 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:30.598197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:30.598207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:30.598265 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:30.598546 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:30.599098 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:30.599961 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:30.600185 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:30.600241 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:30.600449 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:30.600503 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:30.600546 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:30.600608 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:32:30.600627 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260808 00:32:30.603549 3234 raft_consensus.cc:1261] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Refusing update from remote peer 016bfba080a042e2b38f6db8d0ac1616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:30.603592 2837 raft_consensus.cc:1261] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Refusing update from remote peer 016bfba080a042e2b38f6db8d0ac1616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:30.603879 3423 consensus_queue.cc:1035] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:30.603988 3444 consensus_queue.cc:1035] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:30.615149 2598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:30.615258 2598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:30.615952 2598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Sending DeleteTablet for 3 replicas of tablet feb03e3a7da04f1394a788dbcf844af2
I20260808 00:32:30.616266 3004 tablet_service.cc:1514] Processing DeleteTablet for tablet feb03e3a7da04f1394a788dbcf844af2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:47072
I20260808 00:32:30.616297 3209 tablet_service.cc:1514] Processing DeleteTablet for tablet feb03e3a7da04f1394a788dbcf844af2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:53574
I20260808 00:32:30.616351 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet feb03e3a7da04f1394a788dbcf844af2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:37792
I20260808 00:32:30.616416 3462 tablet_replica.cc:330] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: stopping tablet replica
I20260808 00:32:30.616498 3462 raft_consensus.cc:2229] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:30.616463 3461 tablet_replica.cc:330] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: stopping tablet replica
I20260808 00:32:30.616571 3461 raft_consensus.cc:2229] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:30.616578 3462 raft_consensus.cc:2258] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:30.616624 3461 raft_consensus.cc:2258] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:30.616658 3464 tablet_replica.cc:330] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: stopping tablet replica
I20260808 00:32:30.616719 3464 raft_consensus.cc:2229] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:30.616771 3464 raft_consensus.cc:2258] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:30.617008 3461 ts_tablet_manager.cc:1905] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:30.617084 3464 ts_tablet_manager.cc:1905] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:30.617084 2598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:30.617229 2598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:30.618156 3462 ts_tablet_manager.cc:1905] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:30.619248 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 0668046c210a459987af8dd93925bcc8 (DEFAULT_TABLE table=test-table [id=5d498292d57e471483addf99eee6353d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:30.619283 3004 tablet_service.cc:1467] Processing CreateTablet for tablet 0668046c210a459987af8dd93925bcc8 (DEFAULT_TABLE table=test-table [id=5d498292d57e471483addf99eee6353d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:30.619410 3209 tablet_service.cc:1467] Processing CreateTablet for tablet 0668046c210a459987af8dd93925bcc8 (DEFAULT_TABLE table=test-table [id=5d498292d57e471483addf99eee6353d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:30.653841 3004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0668046c210a459987af8dd93925bcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:30.653841 3209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0668046c210a459987af8dd93925bcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:30.655375 3476 tablet_bootstrap.cc:492] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: Bootstrap starting.
I20260808 00:32:30.655697 3474 tablet_bootstrap.cc:492] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: Bootstrap starting.
I20260808 00:32:30.655839 3476 tablet_bootstrap.cc:654] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:30.656073 3474 tablet_bootstrap.cc:654] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:30.656803 3474 tablet_bootstrap.cc:492] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: No bootstrap required, opened a new log
I20260808 00:32:30.656883 3474 ts_tablet_manager.cc:1397] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:30.657074 3474 raft_consensus.cc:348] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.657172 3474 raft_consensus.cc:374] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:30.657208 3474 raft_consensus.cc:729] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a836c8948e49e2a1033839ae41f048, State: Initialized, Role: FOLLOWER
I20260808 00:32:30.657271 3474 consensus_queue.cc:260] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.657310 3464 ts_tablet_manager.cc:1918] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:30.657367 3464 log.cc:1196] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/feb03e3a7da04f1394a788dbcf844af2
I20260808 00:32:30.657398 3474 ts_tablet_manager.cc:1428] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:30.657594 3464 ts_tablet_manager.cc:1939] T feb03e3a7da04f1394a788dbcf844af2 P 46a836c8948e49e2a1033839ae41f048: Deleting consensus metadata
I20260808 00:32:30.659240 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0668046c210a459987af8dd93925bcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:30.659561 3462 ts_tablet_manager.cc:1918] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:30.659633 3462 log.cc:1196] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/feb03e3a7da04f1394a788dbcf844af2
I20260808 00:32:30.660465 3462 ts_tablet_manager.cc:1939] T feb03e3a7da04f1394a788dbcf844af2 P 016bfba080a042e2b38f6db8d0ac1616: Deleting consensus metadata
I20260808 00:32:30.660753 3476 tablet_bootstrap.cc:492] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: No bootstrap required, opened a new log
I20260808 00:32:30.660928 2584 catalog_manager.cc:4841] TS 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791): tablet feb03e3a7da04f1394a788dbcf844af2 (table test-table [id=a62a9d8edac8418c860b2f0c9604fa15]) successfully deleted
I20260808 00:32:30.660840 3476 ts_tablet_manager.cc:1397] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 00:32:30.661124 3476 raft_consensus.cc:348] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.661206 3476 raft_consensus.cc:374] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:30.661224 3476 raft_consensus.cc:729] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Initialized, Role: FOLLOWER
I20260808 00:32:30.661279 3476 consensus_queue.cc:260] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.661383 3476 ts_tablet_manager.cc:1428] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:30.688760 3444 raft_consensus.cc:482] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:30.688874 3444 raft_consensus.cc:504] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.689105 3444 leader_election.cc:288] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:30.689363 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0668046c210a459987af8dd93925bcc8" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048" is_pre_election: true
I20260808 00:32:30.689502 3234 raft_consensus.cc:2454] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 0.
W20260808 00:32:30.689685 2939 leader_election.cc:341] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099): Not found: Tablet not found: 0668046c210a459987af8dd93925bcc8
I20260808 00:32:30.689785 2940 leader_election.cc:302] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 46a836c8948e49e2a1033839ae41f048; no voters: 42a117c33be64331b0eb35e9d6e309b5
I20260808 00:32:30.689883 3444 raft_consensus.cc:2781] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:30.689992 3444 raft_consensus.cc:482] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:30.690027 3444 raft_consensus.cc:3037] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:30.709089 2582 catalog_manager.cc:4841] TS 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661): tablet feb03e3a7da04f1394a788dbcf844af2 (table test-table [id=a62a9d8edac8418c860b2f0c9604fa15]) successfully deleted
I20260808 00:32:30.745288 3461 ts_tablet_manager.cc:1918] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:30.745247 3444 raft_consensus.cc:504] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.745388 3461 log.cc:1196] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/feb03e3a7da04f1394a788dbcf844af2
I20260808 00:32:30.745504 3444 leader_election.cc:288] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 election: Requested vote from peers 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099), 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661)
I20260808 00:32:30.745666 3461 ts_tablet_manager.cc:1939] T feb03e3a7da04f1394a788dbcf844af2 P 42a117c33be64331b0eb35e9d6e309b5: Deleting consensus metadata
W20260808 00:32:30.745970 2939 leader_election.cc:341] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099): Not found: Tablet not found: 0668046c210a459987af8dd93925bcc8
I20260808 00:32:30.746097 2579 catalog_manager.cc:4841] TS 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099): tablet feb03e3a7da04f1394a788dbcf844af2 (table test-table [id=a62a9d8edac8418c860b2f0c9604fa15]) successfully deleted
I20260808 00:32:30.746224 3234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0668046c210a459987af8dd93925bcc8" candidate_uuid: "016bfba080a042e2b38f6db8d0ac1616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46a836c8948e49e2a1033839ae41f048"
I20260808 00:32:30.746320 3234 raft_consensus.cc:3037] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:30.746435 3477 tablet_bootstrap.cc:492] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5: Bootstrap starting.
I20260808 00:32:30.746873 3477 tablet_bootstrap.cc:654] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:30.747268 3234 raft_consensus.cc:2454] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 016bfba080a042e2b38f6db8d0ac1616 in term 1.
I20260808 00:32:30.747475 2940 leader_election.cc:302] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 016bfba080a042e2b38f6db8d0ac1616, 46a836c8948e49e2a1033839ae41f048; no voters: 42a117c33be64331b0eb35e9d6e309b5
I20260808 00:32:30.747686 3444 raft_consensus.cc:2781] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:30.747767 3444 raft_consensus.cc:686] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 LEADER]: Becoming Leader. State: Replica: 016bfba080a042e2b38f6db8d0ac1616, State: Running, Role: LEADER
I20260808 00:32:30.747797 3477 tablet_bootstrap.cc:492] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5: No bootstrap required, opened a new log
I20260808 00:32:30.747822 3444 consensus_queue.cc:237] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.747867 3477 ts_tablet_manager.cc:1397] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:30.748020 3477 raft_consensus.cc:348] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.748113 3477 raft_consensus.cc:374] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:30.748139 3477 raft_consensus.cc:729] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a117c33be64331b0eb35e9d6e309b5, State: Initialized, Role: FOLLOWER
I20260808 00:32:30.748185 3477 consensus_queue.cc:260] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } }
I20260808 00:32:30.748291 3477 ts_tablet_manager.cc:1428] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:30.748426 2598 catalog_manager.cc:5495] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194). New cstate: current_term: 1 leader_uuid: "016bfba080a042e2b38f6db8d0ac1616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "016bfba080a042e2b38f6db8d0ac1616" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:30.771237 3234 raft_consensus.cc:1261] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Refusing update from remote peer 016bfba080a042e2b38f6db8d0ac1616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:30.771207 2837 raft_consensus.cc:3037] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:30.771478 3449 consensus_queue.cc:1035] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46a836c8948e49e2a1033839ae41f048" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:32:30.776382 3432 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 00:32:30.841645 2837 raft_consensus.cc:1261] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5 [term 1 FOLLOWER]: Refusing update from remote peer 016bfba080a042e2b38f6db8d0ac1616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:30.842166 3423 consensus_queue.cc:1035] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a117c33be64331b0eb35e9d6e309b5" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:32.300414 2597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:32.300597 2597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:32.301553 2597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34c092bf9f74480b98637511c4992205: Sending DeleteTablet for 3 replicas of tablet 0668046c210a459987af8dd93925bcc8
I20260808 00:32:32.301860 3209 tablet_service.cc:1514] Processing DeleteTablet for tablet 0668046c210a459987af8dd93925bcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:32 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:53574
I20260808 00:32:32.301849 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet 0668046c210a459987af8dd93925bcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:32 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:37792
I20260808 00:32:32.301846 3004 tablet_service.cc:1514] Processing DeleteTablet for tablet 0668046c210a459987af8dd93925bcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:32 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:47072
I20260808 00:32:32.302179 3539 tablet_replica.cc:330] T 0668046c210a459987af8dd93925bcc8 P 42a117c33be64331b0eb35e9d6e309b5: stopping tablet replica
I20260808 00:32:32.302233 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 2707
I20260808 00:32:32.312990 3537 tablet_replica.cc:330] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: stopping tablet replica
I20260808 00:32:32.313107 3537 raft_consensus.cc:2229] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:32.313221 3537 raft_consensus.cc:2258] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:32.313401 3538 tablet_replica.cc:330] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: stopping tablet replica
I20260808 00:32:32.313484 3538 raft_consensus.cc:2229] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:32.313558 3538 raft_consensus.cc:2258] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:32.313639 3537 ts_tablet_manager.cc:1905] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:32.313895 3538 ts_tablet_manager.cc:1905] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:32.315762 3538 ts_tablet_manager.cc:1918] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:32:32.315853 3538 log.cc:1196] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/0668046c210a459987af8dd93925bcc8
I20260808 00:32:32.315843 3537 ts_tablet_manager.cc:1918] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:32:32.315904 3537 log.cc:1196] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/0668046c210a459987af8dd93925bcc8
I20260808 00:32:32.316139 3538 ts_tablet_manager.cc:1939] T 0668046c210a459987af8dd93925bcc8 P 46a836c8948e49e2a1033839ae41f048: Deleting consensus metadata
I20260808 00:32:32.316217 3537 ts_tablet_manager.cc:1939] T 0668046c210a459987af8dd93925bcc8 P 016bfba080a042e2b38f6db8d0ac1616: Deleting consensus metadata
I20260808 00:32:32.316601 2582 catalog_manager.cc:4841] TS 46a836c8948e49e2a1033839ae41f048 (127.31.210.195:40661): tablet 0668046c210a459987af8dd93925bcc8 (table test-table [id=5d498292d57e471483addf99eee6353d]) successfully deleted
I20260808 00:32:32.316634 2584 catalog_manager.cc:4841] TS 016bfba080a042e2b38f6db8d0ac1616 (127.31.210.194:34791): tablet 0668046c210a459987af8dd93925bcc8 (table test-table [id=5d498292d57e471483addf99eee6353d]) successfully deleted
W20260808 00:32:32.317749 2579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:38099 (error 108)
I20260808 00:32:32.318075 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 2917
I20260808 00:32:32.321723 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:32.318252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38099 (local address 127.0.0.1:36038)
0808 00:32:32.321603 (+ 3351us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:32.321669 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3316,"thread_start_us":62,"threads_started":1}
W20260808 00:32:32.321900 2579 catalog_manager.cc:4584] TS 42a117c33be64331b0eb35e9d6e309b5 (127.31.210.193:38099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0668046c210a459987af8dd93925bcc8: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38099: connect: Connection refused (error 111)
I20260808 00:32:32.327814 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 3110
I20260808 00:32:32.338371 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 2555
2026-08-08T00:32:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6102 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:32 dist-test-slave-44d2 krb5kdc[3546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:32 dist-test-slave-44d2 krb5kdc[3546](info): set up 2 sockets
Aug 08 00:32:32 dist-test-slave-44d2 krb5kdc[3546](info): commencing operation
krb5kdc: starting...
W20260808 00:32:33.589059 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:33 dist-test-slave-44d2 krb5kdc[3546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:33.724664 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:35545
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:35545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:33.817235 3562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:33.817477 3562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:33.817504 3562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:33.818790 3562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:33.818864 3562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:33.818882 3562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:33.818897 3562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:33.818912 3562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:33.820453 3562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:35545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:35545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:33.820777 3562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:33.820996 3562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:33.826798 3570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:33.826820 3562 server_base.cc:1034] running on GCE node
W20260808 00:32:33.827087 3571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:33.826798 3573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:33.827433 3562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:33.827790 3562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:33.829020 3562 hybrid_clock.cc:648] HybridClock initialized: now 1786149153828989 us; error 43 us; skew 500 ppm
Aug 08 00:32:33 dist-test-slave-44d2 krb5kdc[3546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:33.837369 3562 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:33.837865 3562 webserver.cc:460] Webserver started at http://127.31.210.254:39029/ using document root <none> and password file <none>
I20260808 00:32:33.838090 3562 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:33.838140 3562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:33.838249 3562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:33.839426 3562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "82c09d4c60804ef29d22be0bf421d313"
format_stamp: "Formatted at 2026-08-08 00:32:33 on dist-test-slave-44d2"
server_key: "658edf19c9733ccbd4331297ab7dcf90"
server_key_iv: "4fdcd7b328df8e89141a517ddf9b3898"
server_key_version: "encryptionkey@0"
I20260808 00:32:33.839778 3562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "82c09d4c60804ef29d22be0bf421d313"
format_stamp: "Formatted at 2026-08-08 00:32:33 on dist-test-slave-44d2"
server_key: "658edf19c9733ccbd4331297ab7dcf90"
server_key_iv: "4fdcd7b328df8e89141a517ddf9b3898"
server_key_version: "encryptionkey@0"
I20260808 00:32:33.841413 3562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:33.842269 3581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:33.842495 3562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:33.842603 3562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "82c09d4c60804ef29d22be0bf421d313"
format_stamp: "Formatted at 2026-08-08 00:32:33 on dist-test-slave-44d2"
server_key: "658edf19c9733ccbd4331297ab7dcf90"
server_key_iv: "4fdcd7b328df8e89141a517ddf9b3898"
server_key_version: "encryptionkey@0"
I20260808 00:32:33.842669 3562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:33.853360 3562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:33.855939 3562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:33.856076 3562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:33.860631 3562 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:35545
I20260808 00:32:33.860702 3665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:35545 every 8 connection(s)
I20260808 00:32:33.861092 3562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:33.861483 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 3562
I20260808 00:32:33.861627 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:33.861855 32587 external_mini_cluster.cc:1428] Setting key 4fa4f533e35916e1fe1938bd8157e5ba
I20260808 00:32:33.862401 3666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:33.866230 3666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313: Bootstrap starting.
I20260808 00:32:33.866972 3666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:33.867298 3666 log.cc:824] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:33.868055 3666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313: No bootstrap required, opened a new log
Aug 08 00:32:33 dist-test-slave-44d2 krb5kdc[3546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:33.870237 3666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c09d4c60804ef29d22be0bf421d313" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35545 } }
I20260808 00:32:33.870436 3666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:33.870486 3666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c09d4c60804ef29d22be0bf421d313, State: Initialized, Role: FOLLOWER
I20260808 00:32:33.870644 3666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c09d4c60804ef29d22be0bf421d313" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35545 } }
I20260808 00:32:33.870766 3666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:33.870818 3666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:33.870893 3666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:33.871479 3666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c09d4c60804ef29d22be0bf421d313" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35545 } }
I20260808 00:32:33.871608 3666 leader_election.cc:302] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82c09d4c60804ef29d22be0bf421d313; no voters:
I20260808 00:32:33.871824 3666 leader_election.cc:288] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:33.871867 3672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:33.872007 3672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [term 1 LEADER]: Becoming Leader. State: Replica: 82c09d4c60804ef29d22be0bf421d313, State: Running, Role: LEADER
I20260808 00:32:33.872108 3666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:33.872107 3672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c09d4c60804ef29d22be0bf421d313" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35545 } }
I20260808 00:32:33.872730 3675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82c09d4c60804ef29d22be0bf421d313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c09d4c60804ef29d22be0bf421d313" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35545 } } }
I20260808 00:32:33.872849 3675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:33.873951 3685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:33.874202 3674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c09d4c60804ef29d22be0bf421d313. Latest consensus state: current_term: 1 leader_uuid: "82c09d4c60804ef29d22be0bf421d313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c09d4c60804ef29d22be0bf421d313" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35545 } } }
I20260808 00:32:33.874303 3674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:33.874396 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:33.862504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.31.210.254:35545)
0808 00:32:33.862720 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:33.862731 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:33.862770 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:32:33.863239 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:33.863246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:33.863296 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:33.863419 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:33.864302 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:33.865489 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:33.866580 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:33.867045 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:33.867158 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:33.869928 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:33.869933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:33.869933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:33.869944 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:33.872216 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:33.872695 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:33.872699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:33.872699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:33.872834 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:33.873102 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:33.873103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:33.873104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:33.873424 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:33.873635 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:33.873912 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 00:32:33.874074 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":117,"threads_started":1}
I20260808 00:32:33.874660 3685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:33.876394 3685 catalog_manager.cc:1347] Generated new cluster ID: 54019f4fba3d4926863dd09ea7cc6a2d
I20260808 00:32:33.876467 3685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:33.894994 3685 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:33.895567 3685 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:33.900000 3685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82c09d4c60804ef29d22be0bf421d313: Generated new TSK 0
I20260808 00:32:33.900221 3685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:33.937417 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35545
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:34.024101 3701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:34.024305 3701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:34.024329 3701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:34.025538 3701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:34.025583 3701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:34.025609 3701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:34.027057 3701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:34.027302 3701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:34.027565 3701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:34.033003 3710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:34.033188 3701 server_base.cc:1034] running on GCE node
W20260808 00:32:34.033033 3706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:34.033002 3708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:34.033557 3701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:34.033814 3701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:34.035000 3701 hybrid_clock.cc:648] HybridClock initialized: now 1786149154035013 us; error 75 us; skew 500 ppm
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[3546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149154, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:34.037990 3701 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:34.038484 3701 webserver.cc:460] Webserver started at http://127.31.210.193:44255/ using document root <none> and password file <none>
I20260808 00:32:34.038766 3701 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:34.038846 3701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:34.039016 3701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:34.040231 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "37a6e13fc23642fbb459168a5039c46e"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "ef138e47ecdc0b2d9ac2877bdc4c8394"
server_key_iv: "242795dbfd13c204b94b76a4034cc274"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.040679 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "37a6e13fc23642fbb459168a5039c46e"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "ef138e47ecdc0b2d9ac2877bdc4c8394"
server_key_iv: "242795dbfd13c204b94b76a4034cc274"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.042274 3701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:34.043244 3725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.043566 3701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:34.043706 3701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "37a6e13fc23642fbb459168a5039c46e"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "ef138e47ecdc0b2d9ac2877bdc4c8394"
server_key_iv: "242795dbfd13c204b94b76a4034cc274"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.043792 3701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:34.057575 3701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:34.059944 3701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:34.060163 3701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:34.060452 3701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:34.060855 3701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:34.060913 3701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.061003 3701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:34.061035 3701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.067138 3701 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:45215
I20260808 00:32:34.067210 3899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:45215 every 8 connection(s)
I20260808 00:32:34.067570 3701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:34.073486 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 3701
I20260808 00:32:34.073590 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:34.073808 32587 external_mini_cluster.cc:1428] Setting key c539a46dc6f62107b0e8ad51f666a9be
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[3546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149154, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:34.079099 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:34.068552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:39551 (local address 127.31.210.254:35545)
0808 00:32:34.068604 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:34.068605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:34.068887 (+ 282us) server_negotiation.cc:407] Connection header received
0808 00:32:34.069457 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:34.069458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:34.069470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:34.069520 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:34.070544 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.071076 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.072217 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.072435 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.072556 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:34.075448 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:34.075449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:34.075450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:34.075466 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:34.077399 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:34.077836 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:34.077837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:34.077838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:34.077919 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:34.078255 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:34.078256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:34.078257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:34.078425 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:34.078517 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:34.078819 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 00:32:34.078940 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:32:34.079128 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:34.068572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:35545 (local address 127.31.210.193:39551)
0808 00:32:34.068758 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:34.068795 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:34.069339 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:34.069598 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:34.069601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:34.070223 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:34.070425 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:34.070430 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.071181 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.071183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:34.072127 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:34.072130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.072584 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.072585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:34.072744 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:34.073429 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:34.073438 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:34.075308 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:34.077515 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:34.077522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:34.077524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:34.077757 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:34.078038 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:34.078039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:34.078039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:34.078177 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:34.078535 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:34.078542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:34.078670 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 00:32:34.078763 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 00:32:34.078917 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260808 00:32:34.079733 3901 heartbeater.cc:347] Connected to a master server at 127.31.210.254:35545
I20260808 00:32:34.079877 3901 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:34.080092 3901 heartbeater.cc:511] Master 127.31.210.254:35545 requested a full tablet report, sending...
I20260808 00:32:34.080816 3612 ts_manager.cc:194] Registered new tserver with Master: 37a6e13fc23642fbb459168a5039c46e (127.31.210.193:45215)
I20260808 00:32:34.081914 3612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:39551
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:34.108376 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35545
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:34.194412 3913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:34.194638 3913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:34.194661 3913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:34.195770 3913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:34.195807 3913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:34.195833 3913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:34.197327 3913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:34.197530 3913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:34.197710 3913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:34.202943 3922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:34.202919 3921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:34.202926 3924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:34.203336 3913 server_base.cc:1034] running on GCE node
I20260808 00:32:34.203567 3913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:34.203847 3913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:34.205044 3913 hybrid_clock.cc:648] HybridClock initialized: now 1786149154205020 us; error 46 us; skew 500 ppm
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[3546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149154, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:34.207854 3913 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:34.208400 3913 webserver.cc:460] Webserver started at http://127.31.210.194:39015/ using document root <none> and password file <none>
I20260808 00:32:34.208694 3913 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:34.208778 3913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:34.208964 3913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:34.210124 3913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "b457439fcdb54f69839d4788041e994c"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "92897da8ffde9217211c87da8e75d991"
server_key_iv: "9dfc4c0661e01c304c41fddc846de235"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.210536 3913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "b457439fcdb54f69839d4788041e994c"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "92897da8ffde9217211c87da8e75d991"
server_key_iv: "9dfc4c0661e01c304c41fddc846de235"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.212198 3913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:34.213151 3936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.213339 3913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:34.213444 3913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "b457439fcdb54f69839d4788041e994c"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "92897da8ffde9217211c87da8e75d991"
server_key_iv: "9dfc4c0661e01c304c41fddc846de235"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.213546 3913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:34.232072 3913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:34.234668 3913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:34.234845 3913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:34.235113 3913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:34.235432 3913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:34.235495 3913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.235560 3913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:34.235594 3913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.241851 3913 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:40831
I20260808 00:32:34.241899 4106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:40831 every 8 connection(s)
I20260808 00:32:34.242251 3913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:34.244959 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 3913
I20260808 00:32:34.245064 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:34.245278 32587 external_mini_cluster.cc:1428] Setting key b8a35782d5f4b83d0b36adf0a45ff3bb
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[3546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149154, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:34.253999 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:34.243096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:58283 (local address 127.31.210.254:35545)
0808 00:32:34.243154 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:34.243156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:34.243339 (+ 183us) server_negotiation.cc:407] Connection header received
0808 00:32:34.243896 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:34.243897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:34.243907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:34.243972 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:34.245313 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.245903 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.247255 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.247482 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.247562 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:34.250366 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:34.250368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:34.250368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:34.250377 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:34.251964 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:34.252672 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:34.252674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:34.252674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:34.252758 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:34.253113 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:34.253114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:34.253114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:34.253277 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:34.253344 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:34.253755 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 00:32:34.253925 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:32:34.254194 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:34.243006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:35545 (local address 127.31.210.194:58283)
0808 00:32:34.243201 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:34.243252 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:34.243796 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:34.244110 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:34.244114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:34.244878 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:34.245192 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:34.245201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.246018 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.246019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:34.247146 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:34.247149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.247588 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.247590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:34.247728 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:34.248359 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:34.248368 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:34.250250 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:34.252066 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:34.252075 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:34.252078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:34.252463 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:34.252853 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:34.252855 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:34.252855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:34.253022 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:34.253358 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:34.253366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:34.253565 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 00:32:34.253696 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:32:34.253900 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260808 00:32:34.254832 4107 heartbeater.cc:347] Connected to a master server at 127.31.210.254:35545
I20260808 00:32:34.254935 4107 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:34.255177 4107 heartbeater.cc:511] Master 127.31.210.254:35545 requested a full tablet report, sending...
I20260808 00:32:34.255645 3612 ts_manager.cc:194] Registered new tserver with Master: b457439fcdb54f69839d4788041e994c (127.31.210.194:40831)
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
I20260808 00:32:34.256089 3612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:58283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:34.275677 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35545
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:34.363178 4113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:34.363402 4113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:34.363427 4113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:34.364570 4113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:34.364605 4113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:34.364631 4113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:34.366017 4113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:34.366204 4113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:34.366389 4113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:34.371704 4121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:34.371691 4124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:34.371800 4113 server_base.cc:1034] running on GCE node
W20260808 00:32:34.371693 4122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:34.372215 4113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:34.372439 4113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:34.373663 4113 hybrid_clock.cc:648] HybridClock initialized: now 1786149154373654 us; error 50 us; skew 500 ppm
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[3546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149154, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:34.376525 4113 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:34.377004 4113 webserver.cc:460] Webserver started at http://127.31.210.195:34579/ using document root <none> and password file <none>
I20260808 00:32:34.377239 4113 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:34.377310 4113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:34.377471 4113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:34.378553 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "bd834e345e144998974c9d1207992663"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "5a52d4873e6c2cbaa513b2645889dac9"
server_key_iv: "44377c3ad8db01c123f5eb5b262d5401"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.378926 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "bd834e345e144998974c9d1207992663"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "5a52d4873e6c2cbaa513b2645889dac9"
server_key_iv: "44377c3ad8db01c123f5eb5b262d5401"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.380353 4113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:32:34.381232 4135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.381436 4113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:34.381544 4113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "bd834e345e144998974c9d1207992663"
format_stamp: "Formatted at 2026-08-08 00:32:34 on dist-test-slave-44d2"
server_key: "5a52d4873e6c2cbaa513b2645889dac9"
server_key_iv: "44377c3ad8db01c123f5eb5b262d5401"
server_key_version: "encryptionkey@0"
I20260808 00:32:34.381628 4113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:34.391177 4113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:34.393589 4113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:34.393764 4113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:34.394007 4113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:34.394320 4113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:34.394383 4113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.394428 4113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:34.394451 4113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:34.400810 4113 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:44807
I20260808 00:32:34.400887 4298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:44807 every 8 connection(s)
I20260808 00:32:34.401217 4113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:34.402136 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 4113
I20260808 00:32:34.402238 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:34.402436 32587 external_mini_cluster.cc:1428] Setting key 7078fead144606908f39984e72a3f0e3
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[3546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149154, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:34.411042 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:34.402086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:42687 (local address 127.31.210.254:35545)
0808 00:32:34.402159 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:34.402161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:34.402418 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:32:34.403111 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:34.403112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:34.403126 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:34.403172 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:34.404157 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.404784 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.405789 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.405995 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.406082 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:34.408248 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:34.408250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:34.408250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:34.408258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:34.409649 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:34.410059 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:34.410060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:34.410060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:34.410122 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:34.410384 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:34.410385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:34.410386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:34.410504 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:34.410577 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:34.410859 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 00:32:34.410940 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:32:34.411259 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:34.402044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:35545 (local address 127.31.210.195:42687)
0808 00:32:34.402270 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:34.402320 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:34.402994 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:34.403259 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:34.403263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:34.403846 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:34.404058 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:34.404063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.404879 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.404880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:34.405682 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:34.405684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:34.406094 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:34.406095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:34.406214 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:34.406682 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:34.406690 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:34.408151 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:34.409748 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:34.409754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:34.409757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:34.409982 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:34.410193 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:34.410194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:34.410194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:34.410308 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:34.410580 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:34.410588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:34.410727 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 00:32:34.410838 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 00:32:34.411013 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":128,"threads_started":1}
I20260808 00:32:34.411751 4299 heartbeater.cc:347] Connected to a master server at 127.31.210.254:35545
I20260808 00:32:34.411865 4299 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:34.412070 4299 heartbeater.cc:511] Master 127.31.210.254:35545 requested a full tablet report, sending...
I20260808 00:32:34.412462 3612 ts_manager.cc:194] Registered new tserver with Master: bd834e345e144998974c9d1207992663 (127.31.210.195:44807)
I20260808 00:32:34.412928 3612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:42687
I20260808 00:32:34.414345 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:32:34.429518 3669 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:34.428233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.31.210.254:35545)
0808 00:32:34.428336 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:34.428339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:34.428616 (+ 277us) server_negotiation.cc:407] Connection header received
0808 00:32:34.428733 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:34.428734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:34.428753 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:34.428816 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:34.429453 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44722: BlockingRecv error: recv got EOF from 127.0.0.1:44722 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 00:32:34.430600 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 3701
I20260808 00:32:34.443579 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 3913
I20260808 00:32:34.451653 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 4113
I20260808 00:32:34.458307 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 3562
2026-08-08T00:32:34Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2115 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[4326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[4326](info): set up 2 sockets
Aug 08 00:32:34 dist-test-slave-44d2 krb5kdc[4326](info): commencing operation
krb5kdc: starting...
W20260808 00:32:35.743099 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:35 dist-test-slave-44d2 krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:35.880213 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:41805
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:41805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:35.964883 4343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:35.965102 4343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:35.965124 4343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:35.966292 4343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:35.966336 4343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:35.966351 4343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:35.966363 4343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:35.966375 4343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:35.967880 4343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:41805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:41805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:35.968128 4343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:35.968365 4343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:35.973407 4355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:35.973402 4357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:35.973448 4353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:35.973650 4343 server_base.cc:1034] running on GCE node
I20260808 00:32:35.974023 4343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:35.974280 4343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:35.975441 4343 hybrid_clock.cc:648] HybridClock initialized: now 1786149155975420 us; error 36 us; skew 500 ppm
Aug 08 00:32:35 dist-test-slave-44d2 krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149155, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:35.978080 4343 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:35.978549 4343 webserver.cc:460] Webserver started at http://127.31.210.254:37383/ using document root <none> and password file <none>
I20260808 00:32:35.978780 4343 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:35.978827 4343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:35.978964 4343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:35.980032 4343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "30e90074deeb42419efa41fc1d941e13"
format_stamp: "Formatted at 2026-08-08 00:32:35 on dist-test-slave-44d2"
server_key: "06c362c1ea5ea91ad224296fd7f018d0"
server_key_iv: "8e904696729c4fd63a18bf2d9e949e6a"
server_key_version: "encryptionkey@0"
I20260808 00:32:35.980405 4343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "30e90074deeb42419efa41fc1d941e13"
format_stamp: "Formatted at 2026-08-08 00:32:35 on dist-test-slave-44d2"
server_key: "06c362c1ea5ea91ad224296fd7f018d0"
server_key_iv: "8e904696729c4fd63a18bf2d9e949e6a"
server_key_version: "encryptionkey@0"
I20260808 00:32:35.981834 4343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:35.982620 4367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:35.982882 4343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:35.983003 4343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "30e90074deeb42419efa41fc1d941e13"
format_stamp: "Formatted at 2026-08-08 00:32:35 on dist-test-slave-44d2"
server_key: "06c362c1ea5ea91ad224296fd7f018d0"
server_key_iv: "8e904696729c4fd63a18bf2d9e949e6a"
server_key_version: "encryptionkey@0"
I20260808 00:32:35.983100 4343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:35.991873 4343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:35.993968 4343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:35.994140 4343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:35.998193 4343 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:41805
I20260808 00:32:35.998240 4439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:41805 every 8 connection(s)
I20260808 00:32:35.998823 4343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:35.999522 4441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:36.002002 4441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13: Bootstrap starting.
I20260808 00:32:36.002625 4441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:36.002924 4441 log.cc:824] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:36.003640 4441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13: No bootstrap required, opened a new log
I20260808 00:32:36.005853 4441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30e90074deeb42419efa41fc1d941e13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 41805 } }
I20260808 00:32:36.005997 4441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:36.006021 4441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e90074deeb42419efa41fc1d941e13, State: Initialized, Role: FOLLOWER
I20260808 00:32:36.006171 4441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30e90074deeb42419efa41fc1d941e13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 41805 } }
I20260808 00:32:36.006265 4441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:36.006309 4441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:36.006378 4441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:36.006394 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 4343
I20260808 00:32:36.006486 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:36.006678 32587 external_mini_cluster.cc:1428] Setting key 2ce948ebc0748330f80e0345fdda32fa
I20260808 00:32:36.007750 4441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30e90074deeb42419efa41fc1d941e13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 41805 } }
I20260808 00:32:36.007869 4441 leader_election.cc:302] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30e90074deeb42419efa41fc1d941e13; no voters:
I20260808 00:32:36.008028 4441 leader_election.cc:288] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:36.008059 4447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:36.008167 4447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [term 1 LEADER]: Becoming Leader. State: Replica: 30e90074deeb42419efa41fc1d941e13, State: Running, Role: LEADER
I20260808 00:32:36.008302 4441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:36.008311 4447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30e90074deeb42419efa41fc1d941e13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 41805 } }
I20260808 00:32:36.008832 4448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30e90074deeb42419efa41fc1d941e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30e90074deeb42419efa41fc1d941e13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 41805 } } }
I20260808 00:32:36.009132 4448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:36.009052 4450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30e90074deeb42419efa41fc1d941e13. Latest consensus state: current_term: 1 leader_uuid: "30e90074deeb42419efa41fc1d941e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30e90074deeb42419efa41fc1d941e13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 41805 } } }
I20260808 00:32:36.009588 4450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:36.009740 4461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:36.010373 4461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:36.011983 4461 catalog_manager.cc:1347] Generated new cluster ID: 4c65d7a5b58f42c5bd632b9324be9cc3
I20260808 00:32:36.012038 4461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:36 dist-test-slave-44d2 krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:36.019119 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:36.007206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.31.210.254:41805)
0808 00:32:36.007432 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:36.007443 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:36.007482 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:32:36.007604 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:36.007611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:36.007670 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:36.007831 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:36.008677 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:36.010179 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:36.011004 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:36.011537 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:36.011666 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:36.014946 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:36.014951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:36.014951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:36.014964 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:36.017546 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:36.017924 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:36.017930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:36.017930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:36.018028 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:36.018254 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:36.018254 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:36.018255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:36.018508 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:36.018694 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:36.018764 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 00:32:36.018922 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":117,"threads_started":1}
I20260808 00:32:36.020185 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:32:36.021134 4461 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:36.021914 4461 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:36.031591 4461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30e90074deeb42419efa41fc1d941e13: Generated new TSK 0
I20260808 00:32:36.031845 4461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:32:36.035897 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:36.032235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.31.210.254:41805)
0808 00:32:36.032299 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:36.032302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:36.032447 (+ 145us) server_negotiation.cc:407] Connection header received
0808 00:32:36.032564 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:36.032565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:36.032577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:36.032664 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:36.033065 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:36.033566 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:36.034376 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:36.034600 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:36.034677 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:36.035422 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:36.035423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:36.035424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:36.035618 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:36.035634 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:36.035640 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:36.035705 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:36.035781 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 00:32:36.035810 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:32:36.038703 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 4343
2026-08-08T00:32:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1582 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:36 dist-test-slave-44d2 krb5kdc[4489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:36 dist-test-slave-44d2 krb5kdc[4489](info): set up 2 sockets
Aug 08 00:32:36 dist-test-slave-44d2 krb5kdc[4489](info): commencing operation
krb5kdc: starting...
W20260808 00:32:37.300259 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:37 dist-test-slave-44d2 krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:37.429235 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46405
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:37.512463 4509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:37.512777 4509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:37.512817 4509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:37.513981 4509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:37.514058 4509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:37.514084 4509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:37.514107 4509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:37.514137 4509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:37.515625 4509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:37.515916 4509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:37.516187 4509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:37.521250 4518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:37.521220 4520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:37.521231 4517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:37.521507 4509 server_base.cc:1034] running on GCE node
I20260808 00:32:37.521775 4509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:37.522071 4509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:37.523249 4509 hybrid_clock.cc:648] HybridClock initialized: now 1786149157523236 us; error 47 us; skew 500 ppm
Aug 08 00:32:37 dist-test-slave-44d2 krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149157, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:37.526067 4509 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:37.526638 4509 webserver.cc:460] Webserver started at http://127.31.210.254:35965/ using document root <none> and password file <none>
I20260808 00:32:37.526901 4509 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:37.526981 4509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:37.527135 4509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:37.528213 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "cb19c1c626cb4cb399983ef01b48db90"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "638f2d2a2cc57b50705a3ae955f7096d"
server_key_iv: "f0e98b92e39c55fd29484f6484daa067"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.528635 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "cb19c1c626cb4cb399983ef01b48db90"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "638f2d2a2cc57b50705a3ae955f7096d"
server_key_iv: "f0e98b92e39c55fd29484f6484daa067"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.530108 4509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:37.530910 4531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:37.531143 4509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:37.531250 4509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "cb19c1c626cb4cb399983ef01b48db90"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "638f2d2a2cc57b50705a3ae955f7096d"
server_key_iv: "f0e98b92e39c55fd29484f6484daa067"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.531335 4509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:37.553901 4509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:37.556072 4509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:37.556254 4509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:37.560441 4509 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46405
I20260808 00:32:37.560561 4598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46405 every 8 connection(s)
I20260808 00:32:37.560932 4509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:37.562203 4599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:37.564548 4599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90: Bootstrap starting.
I20260808 00:32:37.565047 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 4509
I20260808 00:32:37.565165 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:37.565258 4599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:37.565378 32587 external_mini_cluster.cc:1428] Setting key 49a5070006ef517a5a7010c37fdd2347
I20260808 00:32:37.565616 4599 log.cc:824] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:37.566414 4599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90: No bootstrap required, opened a new log
I20260808 00:32:37.568262 4599 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb19c1c626cb4cb399983ef01b48db90" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46405 } }
I20260808 00:32:37.568413 4599 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:37.568439 4599 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb19c1c626cb4cb399983ef01b48db90, State: Initialized, Role: FOLLOWER
I20260808 00:32:37.568617 4599 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb19c1c626cb4cb399983ef01b48db90" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46405 } }
I20260808 00:32:37.568734 4599 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:37.568781 4599 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:37.568831 4599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:37.569357 4599 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb19c1c626cb4cb399983ef01b48db90" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46405 } }
I20260808 00:32:37.569490 4599 leader_election.cc:302] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb19c1c626cb4cb399983ef01b48db90; no voters:
I20260808 00:32:37.569689 4599 leader_election.cc:288] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:37.569713 4607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:37.569828 4607 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [term 1 LEADER]: Becoming Leader. State: Replica: cb19c1c626cb4cb399983ef01b48db90, State: Running, Role: LEADER
I20260808 00:32:37.569911 4607 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb19c1c626cb4cb399983ef01b48db90" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46405 } }
I20260808 00:32:37.569968 4599 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:37.570279 4609 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb19c1c626cb4cb399983ef01b48db90. Latest consensus state: current_term: 1 leader_uuid: "cb19c1c626cb4cb399983ef01b48db90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb19c1c626cb4cb399983ef01b48db90" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46405 } } }
I20260808 00:32:37.570552 4609 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:37.570467 4608 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb19c1c626cb4cb399983ef01b48db90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb19c1c626cb4cb399983ef01b48db90" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46405 } } }
I20260808 00:32:37.570827 4608 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:37.570952 4622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:37.571489 4622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:32:37 dist-test-slave-44d2 krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:37.574123 4622 catalog_manager.cc:1347] Generated new cluster ID: 1f35550c26a3411c88545df6c097809a
I20260808 00:32:37.574180 4622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:37.576612 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:37.565936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.31.210.254:46405)
0808 00:32:37.566128 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:37.566138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:37.566172 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:32:37.566671 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:37.566678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:37.566728 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:37.566883 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:37.567530 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.568467 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.569366 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.569998 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.570422 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:37.573155 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:37.573159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:37.573159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:37.573170 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:37.574926 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:37.575367 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:37.575370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:37.575370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:37.575466 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:37.575692 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:37.575693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:37.575693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:37.575931 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:37.576118 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:37.576231 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:32:37.576374 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
I20260808 00:32:37.596560 4622 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:37.598003 4622 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:37.603152 4622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb19c1c626cb4cb399983ef01b48db90: Generated new TSK 0
I20260808 00:32:37.603370 4622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:37.643275 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46405
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:37.730247 4646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:37.730465 4646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:37.730490 4646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:37.731674 4646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:37.731719 4646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:37.731745 4646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:37.733469 4646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:37.733721 4646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:37.734031 4646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:37.739362 4657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:37.739552 4654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:37.739548 4655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:37.739768 4646 server_base.cc:1034] running on GCE node
I20260808 00:32:37.740072 4646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:37.740338 4646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:37.741508 4646 hybrid_clock.cc:648] HybridClock initialized: now 1786149157741491 us; error 33 us; skew 500 ppm
Aug 08 00:32:37 dist-test-slave-44d2 krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149157, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:37.744172 4646 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:37.744711 4646 webserver.cc:460] Webserver started at http://127.31.210.193:35021/ using document root <none> and password file <none>
I20260808 00:32:37.744954 4646 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:37.745023 4646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:37.745163 4646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:37.746258 4646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "e516a146a45d4111a7b29d7c59ad3edd"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "92de1db1f51af380b9254351b680fc83"
server_key_iv: "96564e2b6d89479ad3e0fcef65126553"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.746654 4646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "e516a146a45d4111a7b29d7c59ad3edd"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "92de1db1f51af380b9254351b680fc83"
server_key_iv: "96564e2b6d89479ad3e0fcef65126553"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.748085 4646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:37.748937 4666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:37.749174 4646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:37.749289 4646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "e516a146a45d4111a7b29d7c59ad3edd"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "92de1db1f51af380b9254351b680fc83"
server_key_iv: "96564e2b6d89479ad3e0fcef65126553"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.749378 4646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:37.758701 4646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:37.760830 4646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:37.761009 4646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:37.761262 4646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:37.761595 4646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:37.761650 4646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:37.761685 4646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:37.761714 4646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:37.768060 4646 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:44917
I20260808 00:32:37.768115 4830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:44917 every 8 connection(s)
I20260808 00:32:37.768462 4646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:37.769201 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 4646
I20260808 00:32:37.769323 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:37.769541 32587 external_mini_cluster.cc:1428] Setting key b8f4379bdf30d9aa930f697b9caad6a9
Aug 08 00:32:37 dist-test-slave-44d2 krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149157, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
I20260808 00:32:37.780176 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:37.769305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:44157 (local address 127.31.210.254:46405)
0808 00:32:37.769357 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:37.769359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:37.769625 (+ 266us) server_negotiation.cc:407] Connection header received
0808 00:32:37.770438 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:37.770440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:37.770452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:37.770517 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:37.771553 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.772336 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.774012 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.774217 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.774277 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:37.777084 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:37.777086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:37.777087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:37.777094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:37.778602 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:37.779123 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:37.779124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:37.779124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:37.779178 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:37.779480 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:37.779481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:37.779481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:37.779588 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:37.779638 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:37.780015 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 00:32:37.780106 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:32:37.780405 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:37.769301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46405 (local address 127.31.210.193:44157)
0808 00:32:37.769500 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:37.769545 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:37.770320 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:37.770586 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:37.770590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:37.771247 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:37.771456 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:37.771466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.772444 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.772446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:37.773834 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:37.773838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.774306 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.774308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:37.774461 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:37.775001 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:37.775012 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:37.776978 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:37.778694 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:37.778704 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:37.778707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:37.779033 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:37.779250 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:37.779251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:37.779251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:37.779401 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:37.779653 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:37.779660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:37.779833 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 00:32:37.779975 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 00:32:37.780164 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260808 00:32:37.780998 4831 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46405
I20260808 00:32:37.781101 4831 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:37.781255 4831 heartbeater.cc:511] Master 127.31.210.254:46405 requested a full tablet report, sending...
I20260808 00:32:37.781774 4553 ts_manager.cc:194] Registered new tserver with Master: e516a146a45d4111a7b29d7c59ad3edd (127.31.210.193:44917)
I20260808 00:32:37.782502 4553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:44157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:37.800303 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46405
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:37.887074 4840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:37.887317 4840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:37.887355 4840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:37.888552 4840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:37.888622 4840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:37.888666 4840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:37.890106 4840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:37.890372 4840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:37.890614 4840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:37.896126 4846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:37.896162 4852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:37.896126 4849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:37.896472 4840 server_base.cc:1034] running on GCE node
I20260808 00:32:37.896770 4840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:37.897019 4840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:37.898164 4840 hybrid_clock.cc:648] HybridClock initialized: now 1786149157898151 us; error 29 us; skew 500 ppm
Aug 08 00:32:37 dist-test-slave-44d2 krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149157, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:37.900769 4840 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:37.901317 4840 webserver.cc:460] Webserver started at http://127.31.210.194:40653/ using document root <none> and password file <none>
I20260808 00:32:37.901576 4840 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:37.901649 4840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:37.901803 4840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:37.902921 4840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "14a3e051b30e47cd9c688f4df16eaf5f"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "227fd92d80c77522066a35a6bd9b7118"
server_key_iv: "62f9faccd90f2aed18c0d8c950c77625"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.903293 4840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "14a3e051b30e47cd9c688f4df16eaf5f"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "227fd92d80c77522066a35a6bd9b7118"
server_key_iv: "62f9faccd90f2aed18c0d8c950c77625"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.904743 4840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:32:37.905578 4861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:37.905766 4840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:37.905874 4840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "14a3e051b30e47cd9c688f4df16eaf5f"
format_stamp: "Formatted at 2026-08-08 00:32:37 on dist-test-slave-44d2"
server_key: "227fd92d80c77522066a35a6bd9b7118"
server_key_iv: "62f9faccd90f2aed18c0d8c950c77625"
server_key_version: "encryptionkey@0"
I20260808 00:32:37.905948 4840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:37.925011 4840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:37.927250 4840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:37.927433 4840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:37.927698 4840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:37.928041 4840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:37.928098 4840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:37.928134 4840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:37.928179 4840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:37.934612 4840 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:35579
I20260808 00:32:37.934695 5027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:35579 every 8 connection(s)
I20260808 00:32:37.935083 4840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:37.937067 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 4840
I20260808 00:32:37.937209 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:37.937422 32587 external_mini_cluster.cc:1428] Setting key 0855f307aaed5f082c401f8c97b15b32
Aug 08 00:32:37 dist-test-slave-44d2 krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149157, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:37.945230 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:37.935852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:48203 (local address 127.31.210.254:46405)
0808 00:32:37.935921 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:37.935923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:37.936135 (+ 212us) server_negotiation.cc:407] Connection header received
0808 00:32:37.936647 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:37.936648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:37.936658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:37.936705 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:37.938026 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.938542 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.939579 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.939879 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.939959 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:37.942252 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:37.942253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:37.942254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:37.942262 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:37.943734 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:37.944181 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:37.944182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:37.944183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:37.944244 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:37.944522 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:37.944523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:37.944524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:37.944656 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:37.944739 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:37.945043 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 00:32:37.945134 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:32:37.945438 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:37.935835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46405 (local address 127.31.210.194:48203)
0808 00:32:37.936000 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:37.936044 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:37.936549 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:37.936760 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:37.936763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:37.937652 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:37.937929 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:37.937934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.938649 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.938650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:37.939483 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:37.939485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:37.939988 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:37.939989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:37.940113 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:37.940744 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:37.940753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:37.942149 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:37.943841 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:37.943849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:37.943851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:37.944090 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:37.944313 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:37.944314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:37.944315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:37.944434 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:37.944729 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:37.944737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:37.944882 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 00:32:37.945004 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 00:32:37.945187 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260808 00:32:37.945928 5028 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46405
I20260808 00:32:37.946014 5028 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:37.946218 5028 heartbeater.cc:511] Master 127.31.210.254:46405 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
I20260808 00:32:37.946625 4553 ts_manager.cc:194] Registered new tserver with Master: 14a3e051b30e47cd9c688f4df16eaf5f (127.31.210.194:35579)
I20260808 00:32:37.947024 4553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:48203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:37.967193 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46405
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:38.053157 5033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:38.053367 5033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:38.053391 5033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:38.054488 5033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:38.054523 5033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:38.054548 5033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:38.055938 5033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:38.056140 5033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:38.056337 5033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:38.061484 5044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:38.061497 5041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:38.061491 5039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:38.061759 5033 server_base.cc:1034] running on GCE node
I20260808 00:32:38.061997 5033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:38.062256 5033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:38.063457 5033 hybrid_clock.cc:648] HybridClock initialized: now 1786149158063446 us; error 37 us; skew 500 ppm
Aug 08 00:32:38 dist-test-slave-44d2 krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149158, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:38.066107 5033 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:38.066653 5033 webserver.cc:460] Webserver started at http://127.31.210.195:34787/ using document root <none> and password file <none>
I20260808 00:32:38.066905 5033 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:38.066978 5033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:38.067140 5033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:38.068295 5033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "db9cae8a31694973b3d5fdb61550bf6b"
format_stamp: "Formatted at 2026-08-08 00:32:38 on dist-test-slave-44d2"
server_key: "ea2e32f3f30073097924ecacb8ac80bb"
server_key_iv: "f7ffc9f553f8046dab0b1c03036b209f"
server_key_version: "encryptionkey@0"
I20260808 00:32:38.068715 5033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "db9cae8a31694973b3d5fdb61550bf6b"
format_stamp: "Formatted at 2026-08-08 00:32:38 on dist-test-slave-44d2"
server_key: "ea2e32f3f30073097924ecacb8ac80bb"
server_key_iv: "f7ffc9f553f8046dab0b1c03036b209f"
server_key_version: "encryptionkey@0"
I20260808 00:32:38.070340 5033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:38.071199 5053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:38.071417 5033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:38.071563 5033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "db9cae8a31694973b3d5fdb61550bf6b"
format_stamp: "Formatted at 2026-08-08 00:32:38 on dist-test-slave-44d2"
server_key: "ea2e32f3f30073097924ecacb8ac80bb"
server_key_iv: "f7ffc9f553f8046dab0b1c03036b209f"
server_key_version: "encryptionkey@0"
I20260808 00:32:38.071645 5033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:38.088835 5033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:38.091074 5033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:38.091305 5033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:38.091590 5033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:38.091967 5033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:38.092008 5033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:38.092061 5033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:38.092087 5033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:38.099022 5033 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:34087
I20260808 00:32:38.099124 5215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:34087 every 8 connection(s)
I20260808 00:32:38.099459 5033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:38.103513 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 5033
I20260808 00:32:38.103641 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:38.103837 32587 external_mini_cluster.cc:1428] Setting key c00418d9d92a5923530ec6869286aa91
Aug 08 00:32:38 dist-test-slave-44d2 krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149158, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:38.109836 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:38.100288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:45873 (local address 127.31.210.254:46405)
0808 00:32:38.100372 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:38.100374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:38.100587 (+ 213us) server_negotiation.cc:407] Connection header received
0808 00:32:38.101081 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:38.101083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:38.101101 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:38.101173 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:38.102178 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:38.102687 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:38.103971 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:38.104275 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:38.104349 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:38.106742 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:38.106743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:38.106744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:38.106752 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:38.108244 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:38.108794 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:38.108795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:38.108795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:38.108881 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:38.109163 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:38.109164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:38.109165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:38.109287 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:38.109365 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:38.109657 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 00:32:38.109744 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:32:38.110098 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:38.100239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46405 (local address 127.31.210.195:45873)
0808 00:32:38.100437 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:38.100509 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:38.100986 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:38.101253 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:38.101257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:38.101871 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:38.102080 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:38.102085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:38.102775 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:38.102776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:38.103871 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:38.103874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:38.104383 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:38.104385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:38.104610 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:38.105213 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:38.105222 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:38.106641 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:38.108365 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:38.108373 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:38.108375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:38.108698 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:38.108962 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:38.108963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:38.108963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:38.109086 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:38.109356 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:38.109363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:38.109516 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 00:32:38.109621 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 00:32:38.109784 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
I20260808 00:32:38.110610 5217 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46405
I20260808 00:32:38.110709 5217 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:38.110854 5217 heartbeater.cc:511] Master 127.31.210.254:46405 requested a full tablet report, sending...
I20260808 00:32:38.111217 4553 ts_manager.cc:194] Registered new tserver with Master: db9cae8a31694973b3d5fdb61550bf6b (127.31.210.195:34087)
I20260808 00:32:38.111554 4553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:45873
I20260808 00:32:38.115792 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:32:38 dist-test-slave-44d2 krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149158, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 00:32:38 dist-test-slave-44d2 krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149158, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:38.142410 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:38.134738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.31.210.254:46405)
0808 00:32:38.134827 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:38.134835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:38.134902 (+ 67us) server_negotiation.cc:407] Connection header received
0808 00:32:38.134971 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:38.134973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:38.134986 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:38.135044 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:38.135658 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:38.136151 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:38.137001 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:38.137192 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:38.137251 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:38.139582 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:38.139583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:38.139584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:38.139590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:38.141013 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:38.141373 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:38.141375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:38.141375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:38.141450 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:38.141659 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:38.141660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:38.141660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:38.141931 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:38.142047 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:38.142173 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 00:32:38.142252 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 00:32:38.142729 4553 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50026
I20260808 00:32:38.143327 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 4646
I20260808 00:32:38.154238 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 4840
I20260808 00:32:38.169510 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 5033
I20260808 00:32:38.183142 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 4509
2026-08-08T00:32:38Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2146 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:38 dist-test-slave-44d2 krb5kdc[5234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:38 dist-test-slave-44d2 krb5kdc[5234](info): set up 2 sockets
Aug 08 00:32:38 dist-test-slave-44d2 krb5kdc[5234](info): commencing operation
krb5kdc: starting...
W20260808 00:32:39.418649 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:39 dist-test-slave-44d2 krb5kdc[5234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:39.553985 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:33689
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:33689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:39.644809 5253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:39.645011 5253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:39.645033 5253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:39.646306 5253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:39.646351 5253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:39.646374 5253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:39.646426 5253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:39.646445 5253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:39.648015 5253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:33689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:33689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:39.648283 5253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:39.648546 5253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:39.653579 5260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:39.653579 5265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:39.653584 5261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:39.653991 5253 server_base.cc:1034] running on GCE node
I20260808 00:32:39.654305 5253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:39.654568 5253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:39.655716 5253 hybrid_clock.cc:648] HybridClock initialized: now 1786149159655707 us; error 34 us; skew 500 ppm
Aug 08 00:32:39 dist-test-slave-44d2 krb5kdc[5234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149159, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:39.662930 5253 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:39.663365 5253 webserver.cc:460] Webserver started at http://127.31.210.254:34029/ using document root <none> and password file <none>
I20260808 00:32:39.663583 5253 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:39.663630 5253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:39.663722 5253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:39.664884 5253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1"
format_stamp: "Formatted at 2026-08-08 00:32:39 on dist-test-slave-44d2"
server_key: "f8a41548f25320b242826da3d41bb2ee"
server_key_iv: "c2999487024856792299c56fd030c6f2"
server_key_version: "encryptionkey@0"
I20260808 00:32:39.665198 5253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1"
format_stamp: "Formatted at 2026-08-08 00:32:39 on dist-test-slave-44d2"
server_key: "f8a41548f25320b242826da3d41bb2ee"
server_key_iv: "c2999487024856792299c56fd030c6f2"
server_key_version: "encryptionkey@0"
I20260808 00:32:39.666709 5253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:32:39.667459 5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:39.667691 5253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:39.667789 5253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1"
format_stamp: "Formatted at 2026-08-08 00:32:39 on dist-test-slave-44d2"
server_key: "f8a41548f25320b242826da3d41bb2ee"
server_key_iv: "c2999487024856792299c56fd030c6f2"
server_key_version: "encryptionkey@0"
I20260808 00:32:39.667850 5253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:39.692464 5253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:39.694890 5253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:39.695024 5253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:39.699747 5253 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:33689
I20260808 00:32:39.699782 5347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:33689 every 8 connection(s)
I20260808 00:32:39.700403 5253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:39.701251 5348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:39.703684 5348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1: Bootstrap starting.
I20260808 00:32:39.704388 5348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:39.704738 5348 log.cc:824] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:39.705546 5348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1: No bootstrap required, opened a new log
I20260808 00:32:39.707173 5348 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33689 } }
I20260808 00:32:39.707340 5348 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:39.707388 5348 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6f1e5822ac4234b5019bc2a75ec3b1, State: Initialized, Role: FOLLOWER
I20260808 00:32:39.707590 5348 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33689 } }
I20260808 00:32:39.707700 5348 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:39.707739 5348 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:39.707792 5348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:39.708801 5348 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33689 } }
I20260808 00:32:39.708933 5348 leader_election.cc:302] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6f1e5822ac4234b5019bc2a75ec3b1; no voters:
I20260808 00:32:39.709146 5348 leader_election.cc:288] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:39.709203 5351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:39.709419 5348 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:39.709501 5351 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [term 1 LEADER]: Becoming Leader. State: Replica: ac6f1e5822ac4234b5019bc2a75ec3b1, State: Running, Role: LEADER
I20260808 00:32:39.709609 5351 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33689 } }
I20260808 00:32:39.710108 5354 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33689 } } }
I20260808 00:32:39.710115 5355 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6f1e5822ac4234b5019bc2a75ec3b1. Latest consensus state: current_term: 1 leader_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f1e5822ac4234b5019bc2a75ec3b1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33689 } } }
I20260808 00:32:39.710229 5354 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:39.710239 5355 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:39.710556 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 5253
I20260808 00:32:39.710662 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:39.710673 5364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:39.710875 32587 external_mini_cluster.cc:1428] Setting key d28e3f62d8790a9868a84789fe3198c4
I20260808 00:32:39.711349 5364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:39.713280 5364 catalog_manager.cc:1347] Generated new cluster ID: 8a65ad870f6f43358ee863801f287f85
I20260808 00:32:39.713342 5364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:39 dist-test-slave-44d2 krb5kdc[5234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:39.721549 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:39.711705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.31.210.254:33689)
0808 00:32:39.711963 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:39.711973 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:39.712184 (+ 211us) server_negotiation.cc:407] Connection header received
0808 00:32:39.712334 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:39.712340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:39.712385 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:39.712592 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:39.713355 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:39.714289 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:39.715059 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:39.715536 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:39.715660 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:39.718179 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:39.718184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:39.718184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:39.718197 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:39.719921 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:39.720327 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:39.720330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:39.720331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:39.720434 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:39.720681 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:39.720682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:39.720682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:39.720929 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:39.721115 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:39.721186 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 00:32:39.721332 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260808 00:32:39.729602 5364 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:39.730242 5364 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:39.744508 5364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6f1e5822ac4234b5019bc2a75ec3b1: Generated new TSK 0
I20260808 00:32:39.744771 5364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:39.779227 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:33689
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:39.900863 5392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:39.901062 5392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:39.901084 5392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:39.902160 5392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:39.902194 5392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:39.902218 5392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:39.903697 5392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:33689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:39.903882 5392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:39.904057 5392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:39.909902 5401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:39.910090 5399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:39.909926 5392 server_base.cc:1034] running on GCE node
W20260808 00:32:39.909909 5403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:39.910403 5392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:39.910681 5392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:39.911849 5392 hybrid_clock.cc:648] HybridClock initialized: now 1786149159911835 us; error 47 us; skew 500 ppm
Aug 08 00:32:39 dist-test-slave-44d2 krb5kdc[5234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149159, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:39.914362 5392 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:39.914812 5392 webserver.cc:460] Webserver started at http://127.31.210.193:42547/ using document root <none> and password file <none>
I20260808 00:32:39.915055 5392 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:39.915130 5392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:39.915284 5392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:39.916473 5392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "db3c14be725c405fa4b1a4278dcf762b"
format_stamp: "Formatted at 2026-08-08 00:32:39 on dist-test-slave-44d2"
server_key: "3e37f2d6732fabf839feeddc311279f7"
server_key_iv: "b01bc8394d1bbf3ec474d95539bca64b"
server_key_version: "encryptionkey@0"
I20260808 00:32:39.916909 5392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "db3c14be725c405fa4b1a4278dcf762b"
format_stamp: "Formatted at 2026-08-08 00:32:39 on dist-test-slave-44d2"
server_key: "3e37f2d6732fabf839feeddc311279f7"
server_key_iv: "b01bc8394d1bbf3ec474d95539bca64b"
server_key_version: "encryptionkey@0"
I20260808 00:32:39.918308 5392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:39.919077 5415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:39.919250 5392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:39.919358 5392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "db3c14be725c405fa4b1a4278dcf762b"
format_stamp: "Formatted at 2026-08-08 00:32:39 on dist-test-slave-44d2"
server_key: "3e37f2d6732fabf839feeddc311279f7"
server_key_iv: "b01bc8394d1bbf3ec474d95539bca64b"
server_key_version: "encryptionkey@0"
I20260808 00:32:39.919432 5392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:39.944253 5392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:39.946548 5392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:39.946728 5392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:39.947002 5392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:39.947335 5392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:39.947393 5392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:39.947434 5392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:39.947475 5392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:39.953763 5392 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:45337
I20260808 00:32:39.953828 5588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:45337 every 8 connection(s)
I20260808 00:32:39.954211 5392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:39.956446 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 5392
I20260808 00:32:39.956610 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:39.956799 32587 external_mini_cluster.cc:1428] Setting key 141dd8fc590581d213d4c7f61b3853dd
Aug 08 00:32:39 dist-test-slave-44d2 krb5kdc[5234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149159, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:39.964509 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:39.955053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:48157 (local address 127.31.210.254:33689)
0808 00:32:39.955115 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:39.955118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:39.955293 (+ 175us) server_negotiation.cc:407] Connection header received
0808 00:32:39.956243 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:39.956244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:39.956258 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:39.956351 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:39.957296 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:39.957896 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:39.958932 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:39.959158 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:39.959238 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:39.961510 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:39.961511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:39.961512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:39.961519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:39.963024 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:39.963463 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:39.963465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:39.963466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:39.963534 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:39.963803 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:39.963804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:39.963804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:39.963924 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:39.964001 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:39.964293 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 00:32:39.964384 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:32:39.964699 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:39.954983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:33689 (local address 127.31.210.193:48157)
0808 00:32:39.955162 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:39.955205 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:39.956152 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:39.956450 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:39.956454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:39.956842 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:39.957188 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:39.957195 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:39.958010 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:39.958011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:39.958835 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:39.958837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:39.959247 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:39.959248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:39.959400 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:39.959968 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:39.959983 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:39.961401 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:39.963128 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:39.963136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:39.963139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:39.963370 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:39.963609 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:39.963610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:39.963610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:39.963726 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:39.963994 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:39.964001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:39.964149 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 00:32:39.964257 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 00:32:39.964423 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260808 00:32:39.965432 5589 heartbeater.cc:347] Connected to a master server at 127.31.210.254:33689
I20260808 00:32:39.965546 5589 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:39.965749 5589 heartbeater.cc:511] Master 127.31.210.254:33689 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
I20260808 00:32:39.966302 5299 ts_manager.cc:194] Registered new tserver with Master: db3c14be725c405fa4b1a4278dcf762b (127.31.210.193:45337)
I20260808 00:32:39.966940 5299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:48157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:39.985462 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:33689
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:40.070210 5596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:40.070500 5596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:40.070528 5596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:40.071775 5596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:40.071820 5596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:40.071846 5596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:40.073467 5596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:33689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:40.073666 5596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:40.073846 5596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:40.079103 5608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:40.079099 5605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:40.079176 5596 server_base.cc:1034] running on GCE node
W20260808 00:32:40.079077 5604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:40.079552 5596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:40.079757 5596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:40.080854 5596 hybrid_clock.cc:648] HybridClock initialized: now 1786149160080826 us; error 47 us; skew 500 ppm
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[5234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149160, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:40.083472 5596 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:40.083977 5596 webserver.cc:460] Webserver started at http://127.31.210.194:33337/ using document root <none> and password file <none>
I20260808 00:32:40.084244 5596 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:40.084291 5596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:40.084582 5596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:40.085660 5596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "134874edc04748caa21f6db94146467f"
format_stamp: "Formatted at 2026-08-08 00:32:40 on dist-test-slave-44d2"
server_key: "77129500b86ff23686e636099ba29703"
server_key_iv: "189b3cdfce28cc9867541f589d276c2a"
server_key_version: "encryptionkey@0"
I20260808 00:32:40.086040 5596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "134874edc04748caa21f6db94146467f"
format_stamp: "Formatted at 2026-08-08 00:32:40 on dist-test-slave-44d2"
server_key: "77129500b86ff23686e636099ba29703"
server_key_iv: "189b3cdfce28cc9867541f589d276c2a"
server_key_version: "encryptionkey@0"
I20260808 00:32:40.087395 5596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:40.088124 5616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:40.088320 5596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:40.088408 5596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "134874edc04748caa21f6db94146467f"
format_stamp: "Formatted at 2026-08-08 00:32:40 on dist-test-slave-44d2"
server_key: "77129500b86ff23686e636099ba29703"
server_key_iv: "189b3cdfce28cc9867541f589d276c2a"
server_key_version: "encryptionkey@0"
I20260808 00:32:40.088465 5596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:40.101203 5596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:40.104869 5596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:40.105005 5596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:40.105226 5596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:40.105595 5596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:40.105651 5596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:40.105690 5596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:40.105710 5596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:40.112643 5596 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:36011
I20260808 00:32:40.112742 5806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:36011 every 8 connection(s)
I20260808 00:32:40.113181 5596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:40.121522 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 5596
I20260808 00:32:40.121645 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:40.121833 32587 external_mini_cluster.cc:1428] Setting key 5d38bf2a9245d81caccc1c23b188bd29
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[5234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149160, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:40.129482 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:40.114040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:52561 (local address 127.31.210.254:33689)
0808 00:32:40.114134 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:40.114136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:40.114357 (+ 221us) server_negotiation.cc:407] Connection header received
0808 00:32:40.117371 (+ 3014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:40.117373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:40.117389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:40.117466 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:40.120039 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.120623 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.122078 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.122404 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.122509 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:40.125626 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:40.125628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:40.125629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:40.125637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:40.127616 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.128172 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.128174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.128174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.128238 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.128612 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.128613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.128614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.128773 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:40.128856 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:40.129230 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 00:32:40.129332 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 00:32:40.129689 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:40.114012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:33689 (local address 127.31.210.194:52561)
0808 00:32:40.114206 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:40.114263 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:40.117258 (+ 2995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:40.117752 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:40.117758 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:40.118429 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:40.118804 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:40.118812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.120725 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.120727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:40.121988 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:40.121991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.122522 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.122523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:40.122694 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:40.123407 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:40.123419 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:40.125513 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:40.127723 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:40.127731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:40.127734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:40.128086 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:40.128317 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:40.128318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:40.128318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:40.128541 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:40.128851 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:40.128857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:40.129046 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 00:32:40.129169 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 00:32:40.129366 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260808 00:32:40.130663 5807 heartbeater.cc:347] Connected to a master server at 127.31.210.254:33689
I20260808 00:32:40.130792 5807 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:40.131021 5807 heartbeater.cc:511] Master 127.31.210.254:33689 requested a full tablet report, sending...
I20260808 00:32:40.131482 5299 ts_manager.cc:194] Registered new tserver with Master: 134874edc04748caa21f6db94146467f (127.31.210.194:36011)
I20260808 00:32:40.132016 5299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:52561
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:40.152783 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:33689
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:40.248371 5815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:40.248642 5815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:40.248672 5815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:40.249785 5815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:40.249823 5815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:40.249850 5815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:40.251288 5815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:33689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:40.251471 5815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:40.251655 5815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:40.256690 5824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:40.256703 5822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:40.256752 5821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:40.257395 5815 server_base.cc:1034] running on GCE node
I20260808 00:32:40.257615 5815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:40.257871 5815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:40.259027 5815 hybrid_clock.cc:648] HybridClock initialized: now 1786149160259005 us; error 45 us; skew 500 ppm
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[5234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149160, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:40.261708 5815 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:40.262169 5815 webserver.cc:460] Webserver started at http://127.31.210.195:44625/ using document root <none> and password file <none>
I20260808 00:32:40.262398 5815 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:40.262444 5815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:40.262600 5815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:40.263712 5815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "0bf932f7cc5041d6acf25784567edb07"
format_stamp: "Formatted at 2026-08-08 00:32:40 on dist-test-slave-44d2"
server_key: "a8cf0cbdd46318dad6df0ea18fcf74c7"
server_key_iv: "7edc74927cb3cadc69ca451613605acb"
server_key_version: "encryptionkey@0"
I20260808 00:32:40.264079 5815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "0bf932f7cc5041d6acf25784567edb07"
format_stamp: "Formatted at 2026-08-08 00:32:40 on dist-test-slave-44d2"
server_key: "a8cf0cbdd46318dad6df0ea18fcf74c7"
server_key_iv: "7edc74927cb3cadc69ca451613605acb"
server_key_version: "encryptionkey@0"
I20260808 00:32:40.265580 5815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:40.266350 5837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:40.266554 5815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:40.266644 5815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "0bf932f7cc5041d6acf25784567edb07"
format_stamp: "Formatted at 2026-08-08 00:32:40 on dist-test-slave-44d2"
server_key: "a8cf0cbdd46318dad6df0ea18fcf74c7"
server_key_iv: "7edc74927cb3cadc69ca451613605acb"
server_key_version: "encryptionkey@0"
I20260808 00:32:40.266698 5815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:40.275199 5815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:40.277247 5815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:40.277411 5815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:40.277691 5815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:40.278048 5815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:40.278083 5815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:40.278106 5815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:40.278159 5815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:40.285352 5815 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:37537
I20260808 00:32:40.285409 6008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:37537 every 8 connection(s)
I20260808 00:32:40.285750 5815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:40.288447 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 5815
I20260808 00:32:40.288605 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:40.288792 32587 external_mini_cluster.cc:1428] Setting key 82e52697fe4932f0fcf5248ba5e55eed
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[5234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149160, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:40.296207 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:40.286519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:55197 (local address 127.31.210.254:33689)
0808 00:32:40.286589 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:40.286591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:40.286807 (+ 216us) server_negotiation.cc:407] Connection header received
0808 00:32:40.287846 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:40.287847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:40.287862 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:40.287917 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:40.288683 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.289325 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.290400 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.290642 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.290719 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:40.293164 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:40.293166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:40.293166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:40.293175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:40.294677 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.295137 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.295139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.295139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.295207 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.295477 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.295478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.295479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.295631 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:40.295713 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:40.296020 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 00:32:40.296111 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:32:40.296415 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:40.286491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:33689 (local address 127.31.210.195:55197)
0808 00:32:40.286678 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:40.286716 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:40.287722 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:40.288016 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:40.288020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:40.288308 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:40.288590 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:40.288599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.289426 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.289428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:40.290301 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:40.290303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.290726 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.290727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:40.290859 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:40.291469 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:40.291477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:40.293059 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:40.294788 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:40.294797 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:40.294800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:40.295054 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:40.295278 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:40.295279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:40.295279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:40.295403 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:40.295702 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:40.295709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:40.295861 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 00:32:40.295993 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 00:32:40.296168 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260808 00:32:40.296933 6009 heartbeater.cc:347] Connected to a master server at 127.31.210.254:33689
I20260808 00:32:40.297052 6009 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:40.297251 6009 heartbeater.cc:511] Master 127.31.210.254:33689 requested a full tablet report, sending...
I20260808 00:32:40.297622 5299 ts_manager.cc:194] Registered new tserver with Master: 0bf932f7cc5041d6acf25784567edb07 (127.31.210.195:37537)
I20260808 00:32:40.298030 5299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:55197
I20260808 00:32:40.300519 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[5234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[5234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:40.329221 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:40.321149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.31.210.193:45337)
0808 00:32:40.321341 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:40.321347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:40.321359 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:40.321387 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:40.321389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:40.321404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:40.321477 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:40.322110 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.322964 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.323717 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.324077 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.324133 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:40.326387 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:40.326396 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:40.326396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:40.326403 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:40.328168 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.328514 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.328518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.328518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.328589 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.328759 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.328759 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.328760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.328882 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:40.329042 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:40.329079 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 00:32:40.329143 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[5234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:40.337195 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:40.330715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42388 (local address 127.31.210.254:33689)
0808 00:32:40.330788 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:40.330790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:40.330830 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:32:40.330882 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:40.330883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:40.330892 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:40.330936 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:40.331360 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.331827 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.332537 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:40.332747 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:40.332804 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:40.334948 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:40.334949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:40.334949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:40.334956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:40.336168 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.336472 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.336472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.336473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.336577 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:40.336752 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:40.336753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:40.336753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:40.336895 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:40.336970 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:40.337065 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 00:32:40.337129 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 00:32:40.337484 5299 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42388
I20260808 00:32:40.338001 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 5392
I20260808 00:32:40.350399 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 5596
I20260808 00:32:40.363332 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 5815
I20260808 00:32:40.369374 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 5253
2026-08-08T00:32:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2185 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[6022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[6022](info): set up 2 sockets
Aug 08 00:32:40 dist-test-slave-44d2 krb5kdc[6022](info): commencing operation
krb5kdc: starting...
W20260808 00:32:41.619426 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:41 dist-test-slave-44d2 krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:41.751874 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43107
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:43107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:41.837538 6039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:41.837781 6039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:41.837822 6039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:41.839020 6039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:41.839113 6039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:41.839138 6039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:41.839159 6039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:41.839190 6039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:41.840854 6039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:43107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:41.841169 6039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:41.841418 6039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:32:41.847553 6039 server_base.cc:1034] running on GCE node
W20260808 00:32:41.847553 6046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:41.847585 6048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:41.847695 6044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:41.848007 6039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:41.848330 6039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:41.849502 6039 hybrid_clock.cc:648] HybridClock initialized: now 1786149161849488 us; error 27 us; skew 500 ppm
Aug 08 00:32:41 dist-test-slave-44d2 krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149161, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:41.852203 6039 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:41.852586 6039 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:41.852653 6039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:41.852747 6039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:41.853802 6039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "5a25594b198944d9a79a83c03de87d53"
format_stamp: "Formatted at 2026-08-08 00:32:41 on dist-test-slave-44d2"
server_key: "53daf982eaaf9d96c650c0c58ae98b3e"
server_key_iv: "863837263a1c5022d691a0215bf19e37"
server_key_version: "encryptionkey@0"
I20260808 00:32:41.854136 6039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "5a25594b198944d9a79a83c03de87d53"
format_stamp: "Formatted at 2026-08-08 00:32:41 on dist-test-slave-44d2"
server_key: "53daf982eaaf9d96c650c0c58ae98b3e"
server_key_iv: "863837263a1c5022d691a0215bf19e37"
server_key_version: "encryptionkey@0"
I20260808 00:32:41.855468 6039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:41.856256 6061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:41.856442 6039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:41.856570 6039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "5a25594b198944d9a79a83c03de87d53"
format_stamp: "Formatted at 2026-08-08 00:32:41 on dist-test-slave-44d2"
server_key: "53daf982eaaf9d96c650c0c58ae98b3e"
server_key_iv: "863837263a1c5022d691a0215bf19e37"
server_key_version: "encryptionkey@0"
I20260808 00:32:41.856649 6039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:41.879290 6039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:41.881798 6039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:41.882093 6039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:41.886180 6039 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:43107
I20260808 00:32:41.886214 6143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:43107 every 8 connection(s)
I20260808 00:32:41.886554 6039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:41.887706 6145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:41.887902 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 6039
I20260808 00:32:41.888041 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:41.888231 32587 external_mini_cluster.cc:1428] Setting key 79f0d3a8c085b7bcec7aeaefa0c3a114
I20260808 00:32:41.891342 6145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53: Bootstrap starting.
I20260808 00:32:41.892123 6145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:41.892426 6145 log.cc:824] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:41.893214 6145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53: No bootstrap required, opened a new log
Aug 08 00:32:41 dist-test-slave-44d2 krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:41.895794 6145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25594b198944d9a79a83c03de87d53" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43107 } }
I20260808 00:32:41.895965 6145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:41.896018 6145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25594b198944d9a79a83c03de87d53, State: Initialized, Role: FOLLOWER
I20260808 00:32:41.896155 6145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25594b198944d9a79a83c03de87d53" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43107 } }
I20260808 00:32:41.896252 6145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:41.896293 6145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:41.896343 6145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:41.897118 6145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25594b198944d9a79a83c03de87d53" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43107 } }
I20260808 00:32:41.897235 6145 leader_election.cc:302] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a25594b198944d9a79a83c03de87d53; no voters:
I20260808 00:32:41.897430 6145 leader_election.cc:288] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:41.897506 6152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:41.897698 6145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:41.897691 6152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [term 1 LEADER]: Becoming Leader. State: Replica: 5a25594b198944d9a79a83c03de87d53, State: Running, Role: LEADER
I20260808 00:32:41.898142 6152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25594b198944d9a79a83c03de87d53" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43107 } }
I20260808 00:32:41.898833 6154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a25594b198944d9a79a83c03de87d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25594b198944d9a79a83c03de87d53" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43107 } } }
I20260808 00:32:41.898976 6154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:41.899262 6167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:41.899406 6153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a25594b198944d9a79a83c03de87d53. Latest consensus state: current_term: 1 leader_uuid: "5a25594b198944d9a79a83c03de87d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25594b198944d9a79a83c03de87d53" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43107 } } }
I20260808 00:32:41.899488 6153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:41.899817 6167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:41.900352 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:41.888870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.31.210.254:43107)
0808 00:32:41.889033 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:41.889041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:41.889066 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:32:41.889559 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:41.889567 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:41.889616 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:41.889739 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:41.890583 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:41.891595 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:41.892710 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:41.893218 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:41.893367 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:41.895996 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:41.896000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:41.896000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:41.896013 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:41.898156 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:41.898578 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:41.898582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:41.898582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:41.898691 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:41.898927 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:41.898928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:41.898928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:41.899329 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:41.899471 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:41.900015 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 00:32:41.900156 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260808 00:32:41.901654 6167 catalog_manager.cc:1347] Generated new cluster ID: 91807d90faaa4ec380cd5f0a56978b7a
I20260808 00:32:41.901705 6167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:41.909605 6167 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:41.910372 6167 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:41.917030 6167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53: Generated new TSK 0
I20260808 00:32:41.917232 6167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:41.954212 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43107
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:42.048858 6182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:42.049113 6182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:42.049160 6182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:42.050438 6182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:42.050514 6182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:42.050565 6182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:42.052135 6182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:42.052423 6182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:42.052737 6182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:42.058399 6188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.058415 6191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.058432 6189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:42.058712 6182 server_base.cc:1034] running on GCE node
I20260808 00:32:42.058959 6182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:42.059208 6182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:42.060410 6182 hybrid_clock.cc:648] HybridClock initialized: now 1786149162060379 us; error 45 us; skew 500 ppm
Aug 08 00:32:42 dist-test-slave-44d2 krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:42.063217 6182 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:42.063625 6182 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:42.063676 6182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:42.063815 6182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:42.065027 6182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "ee10f6762c369b8b1d91ea60de925b4d"
server_key_iv: "e4ee1a85522b4305f49ac2913fa60538"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.065407 6182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "ee10f6762c369b8b1d91ea60de925b4d"
server_key_iv: "e4ee1a85522b4305f49ac2913fa60538"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.066807 6182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:32:42.067669 6199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.067883 6182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:42.067958 6182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "ee10f6762c369b8b1d91ea60de925b4d"
server_key_iv: "e4ee1a85522b4305f49ac2913fa60538"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.068056 6182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:42.083731 6182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:42.086061 6182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:42.086228 6182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:42.086524 6182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:42.086879 6182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:42.086937 6182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.086982 6182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:42.087013 6182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.092929 6182 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:40015
I20260808 00:32:42.093096 6366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:40015 every 8 connection(s)
I20260808 00:32:42.093326 6182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:32:42 dist-test-slave-44d2 krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:42.101462 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 6182
I20260808 00:32:42.101621 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:42.101814 32587 external_mini_cluster.cc:1428] Setting key c43adc5c061cb1a137bbc04af4b87167
I20260808 00:32:42.104166 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.094200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:53621 (local address 127.31.210.254:43107)
0808 00:32:42.094299 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.094302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.094534 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:32:42.095073 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.095074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.095086 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.095147 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:42.096159 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.096693 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.097947 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.098215 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.098272 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.100829 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:42.100831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:42.100831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:42.100838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:42.102654 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.103139 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.103141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.103141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.103208 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.103433 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.103434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.103434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.103583 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:42.103651 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.103901 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.104019 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 00:32:42.104244 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.094127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43107 (local address 127.31.210.193:53621)
0808 00:32:42.094383 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.094427 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.094945 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.095209 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.095213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.095512 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:42.096069 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.096075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.096775 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.096775 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.097733 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.097736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.098296 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.098297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.098411 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.099097 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:42.099106 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:42.100742 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:42.102761 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:42.102768 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:42.102770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:42.103058 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:42.103273 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:42.103274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:42.103274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:42.103373 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:42.103652 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:42.103658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:42.103772 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.103857 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.104038 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":109,"threads_started":1}
I20260808 00:32:42.104907 6369 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43107
I20260808 00:32:42.105031 6369 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:42.105211 6369 heartbeater.cc:511] Master 127.31.210.254:43107 requested a full tablet report, sending...
I20260808 00:32:42.105727 6088 ts_manager.cc:194] Registered new tserver with Master: 857006bf13ae4f0ea3e1aa81cefcbcdc (127.31.210.193:40015)
I20260808 00:32:42.106341 6088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:53621
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:42.131162 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43107
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:42.216156 6376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:42.216369 6376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:42.216393 6376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:42.217600 6376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:42.217644 6376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:42.217671 6376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:42.219182 6376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:42.219450 6376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:42.219720 6376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:42.225193 6387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.225248 6390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.225175 6388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:42.225332 6376 server_base.cc:1034] running on GCE node
I20260808 00:32:42.225642 6376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:42.225912 6376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:42.227084 6376 hybrid_clock.cc:648] HybridClock initialized: now 1786149162227063 us; error 40 us; skew 500 ppm
Aug 08 00:32:42 dist-test-slave-44d2 krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:42.230126 6376 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:42.230501 6376 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:42.230582 6376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:42.230738 6376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:42.231813 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "781428066c21474aa91675ad68e9c023"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "71e5adb6a56cdf02d27bbf076aebf753"
server_key_iv: "1b7860b2cdff8464b9ac5c792511e42f"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.232205 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "781428066c21474aa91675ad68e9c023"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "71e5adb6a56cdf02d27bbf076aebf753"
server_key_iv: "1b7860b2cdff8464b9ac5c792511e42f"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.233793 6376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:42.234697 6403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.235070 6376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:42.235183 6376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "781428066c21474aa91675ad68e9c023"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "71e5adb6a56cdf02d27bbf076aebf753"
server_key_iv: "1b7860b2cdff8464b9ac5c792511e42f"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.235271 6376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:42.258579 6376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:42.261122 6376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:42.261245 6376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:42.261442 6376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:42.261786 6376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:42.261832 6376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.261857 6376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:42.261873 6376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.268225 6376 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:39287
I20260808 00:32:42.268267 6560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:39287 every 8 connection(s)
I20260808 00:32:42.268601 6376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:32:42 dist-test-slave-44d2 krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:42.277431 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 6376
I20260808 00:32:42.277599 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:42.277817 32587 external_mini_cluster.cc:1428] Setting key 5bcf879c8f46f528f851952d40c1dd79
I20260808 00:32:42.278666 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.269370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:55887 (local address 127.31.210.254:43107)
0808 00:32:42.269442 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.269444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.269701 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:32:42.270293 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.270295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.270306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.270364 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:42.271396 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.271920 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.273068 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.273260 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.273319 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.275561 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:42.275563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:42.275563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:42.275571 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:42.277052 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.277468 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.277470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.277470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.277535 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.277878 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.277879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.277880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.278045 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:42.278136 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.278461 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.278582 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:32:42.278913 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.269356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43107 (local address 127.31.210.194:55887)
0808 00:32:42.269572 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.269610 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.270178 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.270420 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.270423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.270701 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:42.271302 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.271308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.272040 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.272041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.272981 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.272984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.273338 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.273339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.273437 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.273899 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:42.273907 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:42.275390 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:42.277152 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:42.277160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:42.277162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:42.277387 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:42.277630 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:42.277631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:42.277631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:42.277789 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:42.278126 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:42.278135 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:42.278314 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.278452 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.278645 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260808 00:32:42.279486 6561 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43107
I20260808 00:32:42.279603 6561 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:42.279755 6561 heartbeater.cc:511] Master 127.31.210.254:43107 requested a full tablet report, sending...
I20260808 00:32:42.280174 6088 ts_manager.cc:194] Registered new tserver with Master: 781428066c21474aa91675ad68e9c023 (127.31.210.194:39287)
I20260808 00:32:42.280675 6088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:55887
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:42.306089 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43107
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:34069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:42.390759 6571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:42.390983 6571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:42.391009 6571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:42.392102 6571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:42.392135 6571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:42.392161 6571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:42.393646 6571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:34069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:42.393844 6571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:42.394084 6571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:32:42.399433 6571 server_base.cc:1034] running on GCE node
W20260808 00:32:42.399398 6581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.399389 6578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.399411 6584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:42.399852 6571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:42.400121 6571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:42.401290 6571 hybrid_clock.cc:648] HybridClock initialized: now 1786149162401266 us; error 37 us; skew 500 ppm
Aug 08 00:32:42 dist-test-slave-44d2 krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:42.404268 6571 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:42.404675 6571 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:42.404731 6571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:42.404875 6571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:42.405938 6571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "83871138ccf14cdba3526260811b1a2e"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "467bd8446dc5e8bbe3985d29e7a597ab"
server_key_iv: "f19b14a69cdd3de749a9a355145eeb91"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.406319 6571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "83871138ccf14cdba3526260811b1a2e"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "467bd8446dc5e8bbe3985d29e7a597ab"
server_key_iv: "f19b14a69cdd3de749a9a355145eeb91"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.407667 6571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:42.408583 6592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.408762 6571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:42.408866 6571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "83871138ccf14cdba3526260811b1a2e"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "467bd8446dc5e8bbe3985d29e7a597ab"
server_key_iv: "f19b14a69cdd3de749a9a355145eeb91"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.408950 6571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:42.428771 6571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:42.430685 6571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:42.430850 6571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:42.431099 6571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:42.431433 6571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:42.431500 6571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.431547 6571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:42.431581 6571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.438099 6571 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:35959
I20260808 00:32:42.438135 6747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:35959 every 8 connection(s)
I20260808 00:32:42.438465 6571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:42.442150 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 6571
I20260808 00:32:42.442240 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:42.442418 32587 external_mini_cluster.cc:1428] Setting key 6c51f26e47efc291c9b27703cd8fbd81
Aug 08 00:32:42 dist-test-slave-44d2 krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:42.448655 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.439386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:42051 (local address 127.31.210.254:43107)
0808 00:32:42.439446 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.439448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.439680 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:32:42.440243 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.440245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.440258 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.440322 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:42.441373 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.441885 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.443219 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.443489 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.443565 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.445880 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:42.445882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:42.445883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:42.445891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:42.447209 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.447631 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.447632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.447633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.447700 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.447975 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.447976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.447977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.448099 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:42.448175 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.448459 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.448558 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:32:42.448863 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.439372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43107 (local address 127.31.210.195:42051)
0808 00:32:42.439551 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.439587 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.440126 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.440397 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.440406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.440754 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:42.441260 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.441266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.442001 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.442003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.443127 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.443129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.443572 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.443573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.443723 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.444313 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:42.444321 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:42.445780 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:42.447309 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:42.447316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:42.447318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:42.447551 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:42.447770 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:42.447771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:42.447772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:42.447886 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:42.448166 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:42.448173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:42.448316 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.448425 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.448622 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260808 00:32:42.449334 6748 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43107
I20260808 00:32:42.449440 6748 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:42.449599 6748 heartbeater.cc:511] Master 127.31.210.254:43107 requested a full tablet report, sending...
I20260808 00:32:42.449994 6088 ts_manager.cc:194] Registered new tserver with Master: 83871138ccf14cdba3526260811b1a2e (127.31.210.195:35959)
I20260808 00:32:42.450377 6088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:42051
I20260808 00:32:42.454443 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:32:42.462183 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.455860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.31.210.254:43107)
0808 00:32:42.455938 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.455940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.456058 (+ 118us) server_negotiation.cc:407] Connection header received
0808 00:32:42.456136 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.456137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.456148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.456220 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:42.456851 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.457322 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.458087 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.458280 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.458357 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.459607 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:42.459608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:42.459609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:42.459616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:42.460983 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.461351 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.461352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.461352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.461412 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:42.461620 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:42.461621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:42.461621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:42.461773 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:42.461887 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.461998 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.462083 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:32:42.463996 6088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:42.465729 6088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:42.473091 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.468364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:35959 (local address 127.0.0.1:55728)
0808 00:32:42.468503 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.468546 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.468616 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.469206 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.469209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.469212 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:42.469296 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.469298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.470864 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.470865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.471928 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.471930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.472804 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.472805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.472869 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.472874 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.472922 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.472981 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":53,"thread_start_us":70,"threads_started":1}
I20260808 00:32:42.473148 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.468683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.31.210.194:39287)
0808 00:32:42.468969 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.468983 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.469041 (+ 58us) server_negotiation.cc:407] Connection header received
0808 00:32:42.469170 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.469173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.469190 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.469265 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:42.469538 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.470269 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.471718 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.472639 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.472734 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.472802 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.473003 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.473051 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":123,"threads_started":1}
I20260808 00:32:42.473173 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.468416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:39287 (local address 127.0.0.1:34928)
0808 00:32:42.468987 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.468997 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.469072 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.469359 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.469360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.469362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:42.469444 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.469446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.470372 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.470374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.471627 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.471630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.472744 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.472745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.472850 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.472866 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.472980 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.473032 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":474,"threads_started":1}
I20260808 00:32:42.473582 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.468686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.31.210.195:35959)
0808 00:32:42.468987 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.468992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.469008 (+ 16us) server_negotiation.cc:407] Connection header received
0808 00:32:42.469056 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.469058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.469078 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.469397 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:42.469487 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.470748 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.472017 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.472701 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.473346 (+ 645us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.473407 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.473431 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.473501 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":113,"threads_started":1}
I20260808 00:32:42.473830 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.468692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.31.210.193:40015)
0808 00:32:42.468893 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.468899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.469143 (+ 244us) server_negotiation.cc:407] Connection header received
0808 00:32:42.469192 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.469194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.469211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.469286 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:42.469535 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.470783 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.472322 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.473066 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.473137 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.473358 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.473718 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.473766 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260808 00:32:42.474061 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.468389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:40015 (local address 127.0.0.1:42632)
0808 00:32:42.468991 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.469003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.469072 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.469385 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.469385 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.469388 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:42.469446 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.469447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.470903 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.470904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.472229 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.472232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.473396 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.473397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.473548 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.473552 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.473608 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.474003 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":4,"thread_start_us":118,"threads_started":1}
I20260808 00:32:42.474234 6474 tablet_service.cc:1467] Processing CreateTablet for tablet 6268797c2094466789f6829b23a7e12c (DEFAULT_TABLE table=test-table [id=42b9a230947748caadcb3cc0b6028d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:42.474308 6655 tablet_service.cc:1467] Processing CreateTablet for tablet 6268797c2094466789f6829b23a7e12c (DEFAULT_TABLE table=test-table [id=42b9a230947748caadcb3cc0b6028d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:42.474676 6655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6268797c2094466789f6829b23a7e12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:42.474676 6474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6268797c2094466789f6829b23a7e12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:42.474725 6275 tablet_service.cc:1467] Processing CreateTablet for tablet 6268797c2094466789f6829b23a7e12c (DEFAULT_TABLE table=test-table [id=42b9a230947748caadcb3cc0b6028d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:42.475106 6275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6268797c2094466789f6829b23a7e12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:42.477958 6777 tablet_bootstrap.cc:492] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc: Bootstrap starting.
I20260808 00:32:42.478787 6777 tablet_bootstrap.cc:654] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:42.478879 6780 tablet_bootstrap.cc:492] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Bootstrap starting.
I20260808 00:32:42.478961 6779 tablet_bootstrap.cc:492] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Bootstrap starting.
I20260808 00:32:42.479421 6777 log.cc:824] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:42.479646 6780 tablet_bootstrap.cc:654] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:42.479646 6779 tablet_bootstrap.cc:654] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:42.480362 6780 log.cc:824] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:42.480362 6779 log.cc:824] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:42.481277 6777 tablet_bootstrap.cc:492] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc: No bootstrap required, opened a new log
I20260808 00:32:42.481277 6780 tablet_bootstrap.cc:492] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: No bootstrap required, opened a new log
I20260808 00:32:42.481364 6780 ts_tablet_manager.cc:1397] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:32:42.481365 6777 ts_tablet_manager.cc:1397] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:32:42.481390 6779 tablet_bootstrap.cc:492] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: No bootstrap required, opened a new log
I20260808 00:32:42.481464 6779 ts_tablet_manager.cc:1397] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:32:42.482815 6779 raft_consensus.cc:348] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.483024 6779 raft_consensus.cc:374] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:42.483086 6779 raft_consensus.cc:729] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83871138ccf14cdba3526260811b1a2e, State: Initialized, Role: FOLLOWER
I20260808 00:32:42.483237 6779 consensus_queue.cc:260] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.483631 6779 ts_tablet_manager.cc:1428] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:32:42.483644 6748 heartbeater.cc:503] Master 127.31.210.254:43107 was elected leader, sending a full tablet report...
I20260808 00:32:42.483614 6777 raft_consensus.cc:348] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.483665 6780 raft_consensus.cc:348] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.483800 6780 raft_consensus.cc:374] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:42.483795 6777 raft_consensus.cc:374] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:42.483839 6780 raft_consensus.cc:729] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781428066c21474aa91675ad68e9c023, State: Initialized, Role: FOLLOWER
I20260808 00:32:42.483839 6777 raft_consensus.cc:729] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857006bf13ae4f0ea3e1aa81cefcbcdc, State: Initialized, Role: FOLLOWER
I20260808 00:32:42.483958 6777 consensus_queue.cc:260] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.483984 6780 consensus_queue.cc:260] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.484292 6780 ts_tablet_manager.cc:1428] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:42.484292 6777 ts_tablet_manager.cc:1428] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:42.484355 6369 heartbeater.cc:503] Master 127.31.210.254:43107 was elected leader, sending a full tablet report...
I20260808 00:32:42.484411 6561 heartbeater.cc:503] Master 127.31.210.254:43107 was elected leader, sending a full tablet report...
W20260808 00:32:42.519193 6562 tablet.cc:2369] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:42.540997 6785 raft_consensus.cc:482] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:42.541260 6785 raft_consensus.cc:504] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.541925 6785 leader_election.cc:288] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 781428066c21474aa91675ad68e9c023 (127.31.210.194:39287), 857006bf13ae4f0ea3e1aa81cefcbcdc (127.31.210.193:40015)
I20260808 00:32:42.546227 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.542162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:39359 (local address 127.31.210.194:39287)
0808 00:32:42.542271 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.542273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.542284 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:32:42.542383 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.542385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.542399 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.542516 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:42.542708 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.543494 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.545164 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.545828 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.545895 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.545903 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.546109 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.546152 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 00:32:42.546227 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.542354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:49771 (local address 127.31.210.193:40015)
0808 00:32:42.542402 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.542404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.542612 (+ 208us) server_negotiation.cc:407] Connection header received
0808 00:32:42.542658 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.542659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.542688 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.542747 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:42.542881 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.543563 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.545175 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.545752 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.545847 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.545858 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.546109 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.546155 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:32:42.546231 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.542088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:39287 (local address 127.31.210.195:39359)
0808 00:32:42.542186 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.542215 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.542262 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.542540 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.542541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.542543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:42.542627 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.542630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.543633 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.543635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.545032 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.545036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.545910 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.545910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.546000 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.546005 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.546080 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.546130 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":92}
I20260808 00:32:42.546241 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.542172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:40015 (local address 127.31.210.195:49771)
0808 00:32:42.542520 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.542531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.542603 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.542766 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.542767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.542768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:42.542815 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.542817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.543655 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.543655 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.545073 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:42.545074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.545880 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.545882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:42.546018 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.546022 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.546080 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.546130 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":57,"thread_start_us":249,"threads_started":1}
I20260808 00:32:42.546653 6303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6268797c2094466789f6829b23a7e12c" candidate_uuid: "83871138ccf14cdba3526260811b1a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" is_pre_election: true
I20260808 00:32:42.546674 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6268797c2094466789f6829b23a7e12c" candidate_uuid: "83871138ccf14cdba3526260811b1a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781428066c21474aa91675ad68e9c023" is_pre_election: true
I20260808 00:32:42.546829 6303 raft_consensus.cc:2454] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83871138ccf14cdba3526260811b1a2e in term 0.
I20260808 00:32:42.546845 6502 raft_consensus.cc:2454] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83871138ccf14cdba3526260811b1a2e in term 0.
I20260808 00:32:42.547084 6593 leader_election.cc:302] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83871138ccf14cdba3526260811b1a2e, 857006bf13ae4f0ea3e1aa81cefcbcdc; no voters:
I20260808 00:32:42.547238 6785 raft_consensus.cc:2781] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:42.547295 6785 raft_consensus.cc:482] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:42.547324 6785 raft_consensus.cc:3037] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:42.548069 6785 raft_consensus.cc:504] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.548229 6785 leader_election.cc:288] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [CANDIDATE]: Term 1 election: Requested vote from peers 781428066c21474aa91675ad68e9c023 (127.31.210.194:39287), 857006bf13ae4f0ea3e1aa81cefcbcdc (127.31.210.193:40015)
I20260808 00:32:42.548408 6303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6268797c2094466789f6829b23a7e12c" candidate_uuid: "83871138ccf14cdba3526260811b1a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc"
I20260808 00:32:42.548408 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6268797c2094466789f6829b23a7e12c" candidate_uuid: "83871138ccf14cdba3526260811b1a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781428066c21474aa91675ad68e9c023"
I20260808 00:32:42.548550 6502 raft_consensus.cc:3037] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:42.548558 6303 raft_consensus.cc:3037] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:42.549296 6303 raft_consensus.cc:2454] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83871138ccf14cdba3526260811b1a2e in term 1.
I20260808 00:32:42.549373 6502 raft_consensus.cc:2454] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83871138ccf14cdba3526260811b1a2e in term 1.
I20260808 00:32:42.549536 6593 leader_election.cc:302] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83871138ccf14cdba3526260811b1a2e, 857006bf13ae4f0ea3e1aa81cefcbcdc; no voters:
I20260808 00:32:42.549630 6785 raft_consensus.cc:2781] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:42.549901 6785 raft_consensus.cc:686] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 1 LEADER]: Becoming Leader. State: Replica: 83871138ccf14cdba3526260811b1a2e, State: Running, Role: LEADER
I20260808 00:32:42.549994 6785 consensus_queue.cc:237] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } }
I20260808 00:32:42.551030 6088 catalog_manager.cc:5495] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 83871138ccf14cdba3526260811b1a2e (127.31.210.195). New cstate: current_term: 1 leader_uuid: "83871138ccf14cdba3526260811b1a2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83871138ccf14cdba3526260811b1a2e" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:42.565587 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.563098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.31.210.195:35959)
0808 00:32:42.563163 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.563165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.563173 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:42.563230 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.563231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.563252 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.563313 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:42.563454 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.563931 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.564844 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:42.565047 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:42.565100 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:42.565234 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:42.565339 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:42.565403 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.565434 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.565452 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:32:42.567493 6502 raft_consensus.cc:1261] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 1 FOLLOWER]: Refusing update from remote peer 83871138ccf14cdba3526260811b1a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:42.567495 6303 raft_consensus.cc:1261] T 6268797c2094466789f6829b23a7e12c P 857006bf13ae4f0ea3e1aa81cefcbcdc [term 1 FOLLOWER]: Refusing update from remote peer 83871138ccf14cdba3526260811b1a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:42.567862 6789 consensus_queue.cc:1035] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "857006bf13ae4f0ea3e1aa81cefcbcdc" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 40015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:42.568065 6785 consensus_queue.cc:1035] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "781428066c21474aa91675ad68e9c023" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:42.570637 6792 mvcc.cc:205] Tried to move back new op lower bound from 7316066969873494016 to 7316066969805504512. Current Snapshot: MvccSnapshot[applied={T|T < 7316066969873494016}]
I20260808 00:32:42.571290 6794 mvcc.cc:205] Tried to move back new op lower bound from 7316066969873494016 to 7316066969805504512. Current Snapshot: MvccSnapshot[applied={T|T < 7316066969873494016}]
I20260808 00:32:42.573342 6088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:42.573446 6088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:42.574241 6088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a25594b198944d9a79a83c03de87d53: Sending DeleteTablet for 3 replicas of tablet 6268797c2094466789f6829b23a7e12c
I20260808 00:32:42.574616 6474 tablet_service.cc:1514] Processing DeleteTablet for tablet 6268797c2094466789f6829b23a7e12c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:42 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:34928
I20260808 00:32:42.574824 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 6182
I20260808 00:32:42.574965 6275 tablet_service.cc:1514] Processing DeleteTablet for tablet 6268797c2094466789f6829b23a7e12c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:42 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:42632
I20260808 00:32:42.580308 6655 tablet_service.cc:1514] Processing DeleteTablet for tablet 6268797c2094466789f6829b23a7e12c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:42 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:55728
I20260808 00:32:42.580633 6807 tablet_replica.cc:330] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: stopping tablet replica
I20260808 00:32:42.580735 6807 raft_consensus.cc:2229] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:42.580865 6807 raft_consensus.cc:2258] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:42.581048 6806 tablet_replica.cc:330] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: stopping tablet replica
I20260808 00:32:42.581147 6806 raft_consensus.cc:2229] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:42.581240 6806 raft_consensus.cc:2258] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:42.582116 6806 ts_tablet_manager.cc:1905] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:42.583849 6806 ts_tablet_manager.cc:1918] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:42.583930 6806 log.cc:1196] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/6268797c2094466789f6829b23a7e12c
I20260808 00:32:42.584239 6806 ts_tablet_manager.cc:1939] T 6268797c2094466789f6829b23a7e12c P 781428066c21474aa91675ad68e9c023: Deleting consensus metadata
I20260808 00:32:42.584409 6807 ts_tablet_manager.cc:1905] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:42.584741 6065 catalog_manager.cc:4841] TS 781428066c21474aa91675ad68e9c023 (127.31.210.194:39287): tablet 6268797c2094466789f6829b23a7e12c (table test-table [id=42b9a230947748caadcb3cc0b6028d19]) successfully deleted
I20260808 00:32:42.586773 6807 ts_tablet_manager.cc:1918] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:42.586917 6807 log.cc:1196] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/6268797c2094466789f6829b23a7e12c
I20260808 00:32:42.587283 6807 ts_tablet_manager.cc:1939] T 6268797c2094466789f6829b23a7e12c P 83871138ccf14cdba3526260811b1a2e: Deleting consensus metadata
I20260808 00:32:42.587852 6064 catalog_manager.cc:4841] TS 83871138ccf14cdba3526260811b1a2e (127.31.210.195:35959): tablet 6268797c2094466789f6829b23a7e12c (table test-table [id=42b9a230947748caadcb3cc0b6028d19]) successfully deleted
W20260808 00:32:42.589910 6062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:40015 (error 108)
I20260808 00:32:42.590214 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 6376
I20260808 00:32:42.590824 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.590347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:40015 (local address 127.0.0.1:42638)
0808 00:32:42.590739 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.590782 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:40015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369}
W20260808 00:32:42.600703 6062 catalog_manager.cc:4584] TS 857006bf13ae4f0ea3e1aa81cefcbcdc (127.31.210.193:40015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6268797c2094466789f6829b23a7e12c: Network error: Client connection negotiation failed: client connection to 127.31.210.193:40015: connect: Connection refused (error 111)
I20260808 00:32:42.605646 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 6571
I20260808 00:32:42.615552 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 6039
2026-08-08T00:32:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2246 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T00:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:42Z Disabled control of system clock
I20260808 00:32:42.651077 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38129
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:44035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:38129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:32:42.736377 6813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:42.736693 6813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:42.736733 6813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:42.737900 6813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:42.737953 6813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:42.737973 6813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:42.737991 6813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:42.738008 6813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:42.739435 6813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:44035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:38129
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:42.739722 6813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:42.740032 6813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:42.745162 6820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.745203 6821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:42.745445 6813 server_base.cc:1034] running on GCE node
W20260808 00:32:42.745183 6823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:42.745814 6813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:42.746104 6813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:42.747319 6813 hybrid_clock.cc:648] HybridClock initialized: now 1786149162747312 us; error 37 us; skew 500 ppm
I20260808 00:32:42.748737 6813 webserver.cc:460] Webserver started at http://127.31.210.254:42719/ using document root <none> and password file <none>
I20260808 00:32:42.748998 6813 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:42.749089 6813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:42.749243 6813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:42.750353 6813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "034bcaf5bb7846d1b3577f87dcae9dbd"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "603d1e197618199b8300ee75e36b7390"
server_key_iv: "86b006d352f01a8215bdd94287fb8d80"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.750747 6813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "034bcaf5bb7846d1b3577f87dcae9dbd"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "603d1e197618199b8300ee75e36b7390"
server_key_iv: "86b006d352f01a8215bdd94287fb8d80"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.752177 6813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:42.753093 6831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.753274 6813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:42.753391 6813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "034bcaf5bb7846d1b3577f87dcae9dbd"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "603d1e197618199b8300ee75e36b7390"
server_key_iv: "86b006d352f01a8215bdd94287fb8d80"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.753504 6813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:42.757037 6813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:42.757333 6813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:42.757493 6813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:42.761597 6813 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:38129
I20260808 00:32:42.761631 6916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:38129 every 8 connection(s)
I20260808 00:32:42.761992 6813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:42.762696 6917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:42.765704 6917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd: Bootstrap starting.
I20260808 00:32:42.766511 6917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:42.766834 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 6813
I20260808 00:32:42.766942 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:42.767130 32587 external_mini_cluster.cc:1428] Setting key 4a1734335c3233b1a92ac45fc94159ba
I20260808 00:32:42.767558 6917 log.cc:824] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:42.768292 6917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd: No bootstrap required, opened a new log
I20260808 00:32:42.769982 6917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38129 } }
I20260808 00:32:42.770147 6917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:42.770195 6917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034bcaf5bb7846d1b3577f87dcae9dbd, State: Initialized, Role: FOLLOWER
I20260808 00:32:42.770323 6917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38129 } }
I20260808 00:32:42.770423 6917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:42.770474 6917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:42.770524 6917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:42.771065 6917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38129 } }
I20260808 00:32:42.771193 6917 leader_election.cc:302] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 034bcaf5bb7846d1b3577f87dcae9dbd; no voters:
I20260808 00:32:42.771365 6917 leader_election.cc:288] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:42.771458 6927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:42.771618 6927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [term 1 LEADER]: Becoming Leader. State: Replica: 034bcaf5bb7846d1b3577f87dcae9dbd, State: Running, Role: LEADER
I20260808 00:32:42.771641 6917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:42.771713 6927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38129 } }
I20260808 00:32:42.772138 6930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 034bcaf5bb7846d1b3577f87dcae9dbd. Latest consensus state: current_term: 1 leader_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38129 } } }
I20260808 00:32:42.772236 6930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:42.772483 6929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034bcaf5bb7846d1b3577f87dcae9dbd" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38129 } } }
I20260808 00:32:42.772547 6939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:42.772631 6929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:42.773187 6939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:42.774814 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.767714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.31.210.254:38129)
0808 00:32:42.767915 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.767923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.773055 (+ 5132us) server_negotiation.cc:407] Connection header received
0808 00:32:42.773214 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.773221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.773268 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.773770 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:42.774309 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:42.774313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:42.774314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:42.774326 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:42.774369 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:42.774381 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:42.774462 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.774540 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.774603 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260808 00:32:42.775126 6939 catalog_manager.cc:1347] Generated new cluster ID: ef263d6b3b8d4bfebeaa6478e20ac940
I20260808 00:32:42.775192 6939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:42.792706 6939 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:42.792938 6939 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:42.795495 6939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd: Generated new TSK 0
I20260808 00:32:42.796396 6939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:32:42.798944 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38129
--builtin_ntp_servers=127.31.210.212:44035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:32:42.886174 6955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:42.886438 6955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:42.886482 6955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:42.887694 6955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:42.887769 6955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:42.887811 6955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:42.889263 6955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:44035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:42.889544 6955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:42.889848 6955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:42.896186 6963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.896209 6966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:42.896181 6964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:42.896713 6955 server_base.cc:1034] running on GCE node
I20260808 00:32:42.896998 6955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:42.897293 6955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:42.898494 6955 hybrid_clock.cc:648] HybridClock initialized: now 1786149162898467 us; error 43 us; skew 500 ppm
I20260808 00:32:42.900012 6955 webserver.cc:460] Webserver started at http://127.31.210.193:36681/ using document root <none> and password file <none>
I20260808 00:32:42.900279 6955 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:42.900329 6955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:42.900552 6955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:42.901705 6955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "503812825acd4fd7b7f53b55bb3dfc34"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "c7cf26d3bcd021df53d6fbd87d43802f"
server_key_iv: "1005e6e1a0966ddd8489fb7d42b790a4"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.902153 6955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "503812825acd4fd7b7f53b55bb3dfc34"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "c7cf26d3bcd021df53d6fbd87d43802f"
server_key_iv: "1005e6e1a0966ddd8489fb7d42b790a4"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.903556 6955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:42.904351 6973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.904603 6955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:42.904707 6955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "503812825acd4fd7b7f53b55bb3dfc34"
format_stamp: "Formatted at 2026-08-08 00:32:42 on dist-test-slave-44d2"
server_key: "c7cf26d3bcd021df53d6fbd87d43802f"
server_key_iv: "1005e6e1a0966ddd8489fb7d42b790a4"
server_key_version: "encryptionkey@0"
I20260808 00:32:42.904789 6955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:42.908257 6955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:42.908576 6955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:42.908731 6955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:42.908965 6955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:42.909289 6955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:42.909324 6955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.909380 6955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:42.909407 6955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:42.915369 6955 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:46813
I20260808 00:32:42.915453 7131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:46813 every 8 connection(s)
I20260808 00:32:42.915784 6955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:42.919360 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.916655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:41377 (local address 127.31.210.254:38129)
0808 00:32:42.916747 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:42.916750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:42.917033 (+ 283us) server_negotiation.cc:407] Connection header received
0808 00:32:42.917508 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:42.917510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:42.917517 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:42.917585 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:42.918985 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:42.918987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:42.918987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:42.918992 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:42.919010 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:42.919016 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:42.919090 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:42.919275 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 00:32:42.919304 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 00:32:42.919548 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:42.916673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38129 (local address 127.31.210.193:41377)
0808 00:32:42.916870 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:42.916904 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:42.917400 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:42.917641 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:42.917644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:42.917653 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:42.918731 (+ 1078us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:42.918732 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:42.918762 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:42.918793 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:42.919145 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:42.919152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:42.919155 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:42.919239 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:32:42.919295 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260808 00:32:42.920250 7135 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38129
I20260808 00:32:42.920338 7135 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:42.920611 7135 heartbeater.cc:511] Master 127.31.210.254:38129 requested a full tablet report, sending...
I20260808 00:32:42.921281 6849 ts_manager.cc:194] Registered new tserver with Master: 503812825acd4fd7b7f53b55bb3dfc34 (127.31.210.193:46813)
I20260808 00:32:42.925109 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 6955
I20260808 00:32:42.925230 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:42.925441 32587 external_mini_cluster.cc:1428] Setting key ede50cf996fa0bf579fcd1f25769aa05
I20260808 00:32:42.926477 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38129
--builtin_ntp_servers=127.31.210.212:44035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:32:43.012984 7138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:43.013200 7138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:43.013223 7138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:43.014416 7138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:43.014458 7138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:43.014484 7138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:43.015853 7138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:44035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:43.016113 7138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:43.016426 7138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:43.021574 7148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.021588 7152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:43.021739 7138 server_base.cc:1034] running on GCE node
W20260808 00:32:43.021590 7146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:43.022058 7138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:43.022310 7138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:43.023455 7138 hybrid_clock.cc:648] HybridClock initialized: now 1786149163023432 us; error 37 us; skew 500 ppm
I20260808 00:32:43.024966 7138 webserver.cc:460] Webserver started at http://127.31.210.194:35751/ using document root <none> and password file <none>
I20260808 00:32:43.025218 7138 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:43.025267 7138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:43.025446 7138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:43.026531 7138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "6d015a7923404a419d252c5747b7a1c6"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "af451f0621443205510f09ba52313818"
server_key_iv: "acb5a9f67d772772d2fe1d812c693569"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.026906 7138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "6d015a7923404a419d252c5747b7a1c6"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "af451f0621443205510f09ba52313818"
server_key_iv: "acb5a9f67d772772d2fe1d812c693569"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.028263 7138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:43.029129 7162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.029328 7138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:43.029405 7138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "6d015a7923404a419d252c5747b7a1c6"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "af451f0621443205510f09ba52313818"
server_key_iv: "acb5a9f67d772772d2fe1d812c693569"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.029502 7138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:43.032927 7138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:43.033188 7138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:43.033319 7138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:43.033542 7138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:43.033870 7138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:43.033926 7138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.033967 7138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:43.033994 7138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.040344 7138 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:43351
I20260808 00:32:43.040397 7342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:43351 every 8 connection(s)
I20260808 00:32:43.040781 7138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:43.042304 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 7138
I20260808 00:32:43.042405 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:43.042636 32587 external_mini_cluster.cc:1428] Setting key 856f352c0b6e182f7b252390781b1232
I20260808 00:32:43.043777 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38129
--builtin_ntp_servers=127.31.210.212:44035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 00:32:43.044797 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.041674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:51255 (local address 127.31.210.254:38129)
0808 00:32:43.041757 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.041760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.041992 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:32:43.042669 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.042670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.042678 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.042758 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.044367 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.044369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.044369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.044375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.044398 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.044403 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.044501 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.044673 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.044691 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:32:43.044955 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.041515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38129 (local address 127.31.210.194:51255)
0808 00:32:43.041844 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.041896 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.042547 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.042873 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.042877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.042887 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:43.044169 (+ 1282us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:43.044169 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:43.044204 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:43.044238 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:43.044556 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:43.044564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:43.044568 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:43.044657 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:32:43.044721 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":105,"threads_started":1}
I20260808 00:32:43.045543 7343 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38129
I20260808 00:32:43.045670 7343 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:43.045866 7343 heartbeater.cc:511] Master 127.31.210.254:38129 requested a full tablet report, sending...
I20260808 00:32:43.046335 6849 ts_manager.cc:194] Registered new tserver with Master: 6d015a7923404a419d252c5747b7a1c6 (127.31.210.194:43351)
W20260808 00:32:43.131124 7349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:43.131335 7349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:43.131358 7349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:43.132737 7349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:43.132783 7349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:43.132809 7349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:43.134233 7349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:44035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:43.134488 7349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:43.134796 7349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:43.140303 7362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.140336 7358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.140363 7359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:43.140592 7349 server_base.cc:1034] running on GCE node
I20260808 00:32:43.140825 7349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:43.141094 7349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:43.142297 7349 hybrid_clock.cc:648] HybridClock initialized: now 1786149163142289 us; error 41 us; skew 500 ppm
I20260808 00:32:43.143690 7349 webserver.cc:460] Webserver started at http://127.31.210.195:39693/ using document root <none> and password file <none>
I20260808 00:32:43.143941 7349 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:43.144070 7349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:43.144229 7349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:43.145375 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "ee0993578c7246e89854f4c838d4d946"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "048a6b2e28f6cbe2eb602fd512aa3f7a"
server_key_iv: "f55e4cc40fef77e3039095f036ff7478"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.145749 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "ee0993578c7246e89854f4c838d4d946"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "048a6b2e28f6cbe2eb602fd512aa3f7a"
server_key_iv: "f55e4cc40fef77e3039095f036ff7478"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.147118 7349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:32:43.147864 7371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.148097 7349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:43.148203 7349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "ee0993578c7246e89854f4c838d4d946"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "048a6b2e28f6cbe2eb602fd512aa3f7a"
server_key_iv: "f55e4cc40fef77e3039095f036ff7478"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.148288 7349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:43.151786 7349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:43.152058 7349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:43.152204 7349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:43.152418 7349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:43.152788 7349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:43.152846 7349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.152887 7349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:43.152915 7349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.159205 7349 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:34189
I20260808 00:32:43.159269 7539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:34189 every 8 connection(s)
I20260808 00:32:43.159592 7349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:43.160216 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 7349
I20260808 00:32:43.160337 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:43.160586 32587 external_mini_cluster.cc:1428] Setting key 2ea0410402dce1c8c14a05ff38801550
I20260808 00:32:43.163087 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.160378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:45197 (local address 127.31.210.254:38129)
0808 00:32:43.160428 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.160430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.160735 (+ 305us) server_negotiation.cc:407] Connection header received
0808 00:32:43.161539 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.161541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.161547 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.161606 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.162896 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.162897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.162897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.162901 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.162915 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.162918 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.162972 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.163020 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.163033 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:32:43.163349 7543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.160366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38129 (local address 127.31.210.195:45197)
0808 00:32:43.160595 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.160652 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.161319 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.161704 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.161708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.161719 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:43.162747 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:43.162747 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:43.162778 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:43.162808 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:43.162972 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:43.162974 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:43.162977 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:43.163028 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:32:43.163077 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":119,"threads_started":1}
I20260808 00:32:43.163957 7540 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38129
I20260808 00:32:43.164103 7540 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:43.164274 7540 heartbeater.cc:511] Master 127.31.210.254:38129 requested a full tablet report, sending...
I20260808 00:32:43.164839 6849 ts_manager.cc:194] Registered new tserver with Master: ee0993578c7246e89854f4c838d4d946 (127.31.210.195:34189)
I20260808 00:32:43.172610 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:32:43.175208 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.174162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.31.210.254:38129)
0808 00:32:43.174217 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.174219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.174282 (+ 63us) server_negotiation.cc:407] Connection header received
0808 00:32:43.174338 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.174339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.174344 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.174393 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.175040 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.175041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.175041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.175045 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.175061 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.175065 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.175110 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.175142 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.175156 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:32:43.176831 6849 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:43.178371 6849 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:43.182472 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.180539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:43351 (local address 127.0.0.1:54676)
0808 00:32:43.180722 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.180748 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.180804 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.181429 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.181430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.181434 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:43.181941 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:43.181942 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:43.181971 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:43.181998 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:43.182283 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:43.182291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:43.182294 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:43.182367 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:32:43.182382 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260808 00:32:43.182534 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.180906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.31.210.194:43351)
0808 00:32:43.181108 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.181113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.181124 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:32:43.181151 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.181153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.181162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.181592 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.182134 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.182138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.182139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.182143 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.182177 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.182203 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.182266 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.182403 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.182422 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":81,"threads_started":1}
I20260808 00:32:43.182586 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.180762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34189 (local address 127.0.0.1:51756)
0808 00:32:43.181443 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.181453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.181506 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.181771 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.181773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.181776 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:43.182112 (+ 336us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:43.182112 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:43.182121 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:43.182133 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:43.182424 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:43.182425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:43.182426 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:32:43.182493 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 00:32:43.182506 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":68,"threads_started":1}
I20260808 00:32:43.182907 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.181192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.31.210.195:34189)
0808 00:32:43.181623 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.181627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.181636 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:43.181662 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.181664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.181672 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.181751 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.182231 (+ 480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.182280 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.182280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.182286 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.182324 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.182350 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.182420 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.182655 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.182673 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":154,"threads_started":1}
I20260808 00:32:43.183169 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.180539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46813 (local address 127.0.0.1:45042)
0808 00:32:43.180702 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.180748 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.180807 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.181111 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.181118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.181125 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:43.182473 (+ 1348us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:43.182474 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:43.182480 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:43.182492 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:43.183038 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:43.183039 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:43.183040 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:32:43.183086 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:32:43.183099 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260808 00:32:43.183423 7556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.180699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.31.210.193:46813)
0808 00:32:43.180905 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.180910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.180968 (+ 58us) server_negotiation.cc:407] Connection header received
0808 00:32:43.180993 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.180994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.181003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.181096 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.182806 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.182819 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.182820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.182828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.182859 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.182875 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.182956 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.183325 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.183342 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260808 00:32:43.183400 7240 tablet_service.cc:1467] Processing CreateTablet for tablet d76fb3d5c3184ef5b63a26832d670dce (DEFAULT_TABLE table=test-table [id=46a75c8df6e14c29a076f577700bc603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:43.183602 7438 tablet_service.cc:1467] Processing CreateTablet for tablet d76fb3d5c3184ef5b63a26832d670dce (DEFAULT_TABLE table=test-table [id=46a75c8df6e14c29a076f577700bc603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:43.183846 7240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76fb3d5c3184ef5b63a26832d670dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:43.183977 7438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76fb3d5c3184ef5b63a26832d670dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:43.184175 7042 tablet_service.cc:1467] Processing CreateTablet for tablet d76fb3d5c3184ef5b63a26832d670dce (DEFAULT_TABLE table=test-table [id=46a75c8df6e14c29a076f577700bc603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:43.184556 7042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76fb3d5c3184ef5b63a26832d670dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:43.186743 7563 tablet_bootstrap.cc:492] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Bootstrap starting.
I20260808 00:32:43.187351 7564 tablet_bootstrap.cc:492] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Bootstrap starting.
I20260808 00:32:43.187547 7563 tablet_bootstrap.cc:654] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:43.187894 7563 log.cc:824] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:43.187920 7565 tablet_bootstrap.cc:492] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34: Bootstrap starting.
I20260808 00:32:43.188130 7564 tablet_bootstrap.cc:654] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:43.188459 7564 log.cc:824] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:43.188781 7565 tablet_bootstrap.cc:654] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:43.189109 7565 log.cc:824] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:43.189255 7564 tablet_bootstrap.cc:492] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: No bootstrap required, opened a new log
I20260808 00:32:43.189334 7564 ts_tablet_manager.cc:1397] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:43.189900 7563 tablet_bootstrap.cc:492] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: No bootstrap required, opened a new log
I20260808 00:32:43.189985 7563 ts_tablet_manager.cc:1397] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 00:32:43.190037 7565 tablet_bootstrap.cc:492] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34: No bootstrap required, opened a new log
I20260808 00:32:43.190110 7565 ts_tablet_manager.cc:1397] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:43.192103 7565 raft_consensus.cc:348] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.192068 7564 raft_consensus.cc:348] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.192261 7564 raft_consensus.cc:374] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:43.192261 7565 raft_consensus.cc:374] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:43.192314 7565 raft_consensus.cc:729] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 503812825acd4fd7b7f53b55bb3dfc34, State: Initialized, Role: FOLLOWER
I20260808 00:32:43.192315 7564 raft_consensus.cc:729] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d015a7923404a419d252c5747b7a1c6, State: Initialized, Role: FOLLOWER
I20260808 00:32:43.192469 7564 consensus_queue.cc:260] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.192456 7565 consensus_queue.cc:260] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.192800 7565 ts_tablet_manager.cc:1428] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:43.192806 7564 ts_tablet_manager.cc:1428] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260808 00:32:43.192713 7563 raft_consensus.cc:348] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.192862 7563 raft_consensus.cc:374] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:43.192902 7563 raft_consensus.cc:729] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0993578c7246e89854f4c838d4d946, State: Initialized, Role: FOLLOWER
I20260808 00:32:43.192967 7343 heartbeater.cc:503] Master 127.31.210.254:38129 was elected leader, sending a full tablet report...
I20260808 00:32:43.193045 7563 consensus_queue.cc:260] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.193264 7135 heartbeater.cc:503] Master 127.31.210.254:38129 was elected leader, sending a full tablet report...
I20260808 00:32:43.193367 7540 heartbeater.cc:503] Master 127.31.210.254:38129 was elected leader, sending a full tablet report...
I20260808 00:32:43.193347 7563 ts_tablet_manager.cc:1428] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:32:43.217814 7574 raft_consensus.cc:482] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:43.217953 7574 raft_consensus.cc:504] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.218398 7574 leader_election.cc:288] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 503812825acd4fd7b7f53b55bb3dfc34 (127.31.210.193:46813), ee0993578c7246e89854f4c838d4d946 (127.31.210.195:34189)
I20260808 00:32:43.219892 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.218563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46813 (local address 127.31.210.194:33141)
0808 00:32:43.218632 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.218644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.218687 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.218943 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.218945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.218947 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:43.219470 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:43.219470 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:43.219512 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:43.219525 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:43.219734 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:43.219736 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:43.219737 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:32:43.219796 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:32:43.219813 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 00:32:43.219890 7556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.218721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:33141 (local address 127.31.210.193:46813)
0808 00:32:43.218796 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.218798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.218808 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:43.218838 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.218839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.218845 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.218909 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.219624 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.219629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.219630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.219635 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.219652 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.219656 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.219734 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.219801 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.219814 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:32:43.219906 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.218657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:38129 (local address 127.31.210.195:34189)
0808 00:32:43.218717 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.218719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.219111 (+ 392us) server_negotiation.cc:407] Connection header received
0808 00:32:43.219148 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.219149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.219154 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.219216 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.219691 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.219693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.219693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.219698 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.219714 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.219718 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.219768 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.219820 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.219834 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:32:43.219938 7580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.218564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34189 (local address 127.31.210.194:38129)
0808 00:32:43.218967 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.219022 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.219082 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.219227 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.219227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.219230 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:43.219605 (+ 375us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:32:43.219605 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:43.219610 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:32:43.219630 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:43.219764 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:43.219765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:43.219765 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 00:32:43.219819 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:32:43.219834 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":190,"thread_start_us":134,"threads_started":1}
I20260808 00:32:43.220315 7071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76fb3d5c3184ef5b63a26832d670dce" candidate_uuid: "6d015a7923404a419d252c5747b7a1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503812825acd4fd7b7f53b55bb3dfc34" is_pre_election: true
I20260808 00:32:43.220542 7071 raft_consensus.cc:2454] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d015a7923404a419d252c5747b7a1c6 in term 0.
I20260808 00:32:43.220579 7467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76fb3d5c3184ef5b63a26832d670dce" candidate_uuid: "6d015a7923404a419d252c5747b7a1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0993578c7246e89854f4c838d4d946" is_pre_election: true
I20260808 00:32:43.220759 7467 raft_consensus.cc:2454] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d015a7923404a419d252c5747b7a1c6 in term 0.
I20260808 00:32:43.220798 7165 leader_election.cc:302] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 503812825acd4fd7b7f53b55bb3dfc34, 6d015a7923404a419d252c5747b7a1c6; no voters:
I20260808 00:32:43.220917 7574 raft_consensus.cc:2781] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:43.220971 7574 raft_consensus.cc:482] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:43.221021 7574 raft_consensus.cc:3037] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:43.221813 7574 raft_consensus.cc:504] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.221963 7574 leader_election.cc:288] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [CANDIDATE]: Term 1 election: Requested vote from peers 503812825acd4fd7b7f53b55bb3dfc34 (127.31.210.193:46813), ee0993578c7246e89854f4c838d4d946 (127.31.210.195:34189)
I20260808 00:32:43.222190 7467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76fb3d5c3184ef5b63a26832d670dce" candidate_uuid: "6d015a7923404a419d252c5747b7a1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0993578c7246e89854f4c838d4d946"
I20260808 00:32:43.222172 7071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76fb3d5c3184ef5b63a26832d670dce" candidate_uuid: "6d015a7923404a419d252c5747b7a1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503812825acd4fd7b7f53b55bb3dfc34"
I20260808 00:32:43.222265 7467 raft_consensus.cc:3037] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:43.222265 7071 raft_consensus.cc:3037] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:43.223187 7467 raft_consensus.cc:2454] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d015a7923404a419d252c5747b7a1c6 in term 1.
I20260808 00:32:43.223187 7071 raft_consensus.cc:2454] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d015a7923404a419d252c5747b7a1c6 in term 1.
I20260808 00:32:43.223361 7164 leader_election.cc:302] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d015a7923404a419d252c5747b7a1c6, ee0993578c7246e89854f4c838d4d946; no voters:
I20260808 00:32:43.223472 7574 raft_consensus.cc:2781] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:43.223670 7574 raft_consensus.cc:686] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 1 LEADER]: Becoming Leader. State: Replica: 6d015a7923404a419d252c5747b7a1c6, State: Running, Role: LEADER
I20260808 00:32:43.223767 7574 consensus_queue.cc:237] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } }
I20260808 00:32:43.224622 6847 catalog_manager.cc:5495] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d015a7923404a419d252c5747b7a1c6 (127.31.210.194). New cstate: current_term: 1 leader_uuid: "6d015a7923404a419d252c5747b7a1c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d015a7923404a419d252c5747b7a1c6" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:43.231747 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.230810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.31.210.194:43351)
0808 00:32:43.230874 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.230876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.230885 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:43.230964 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.230965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.230978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.231035 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.231523 (+ 488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.231524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.231524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.231528 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.231542 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.231546 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.231590 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.231634 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.231645 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:32:43.233700 7071 raft_consensus.cc:1261] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 1 FOLLOWER]: Refusing update from remote peer 6d015a7923404a419d252c5747b7a1c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:43.233700 7467 raft_consensus.cc:1261] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 1 FOLLOWER]: Refusing update from remote peer 6d015a7923404a419d252c5747b7a1c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:43.234081 7581 consensus_queue.cc:1035] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "503812825acd4fd7b7f53b55bb3dfc34" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:43.234213 7574 consensus_queue.cc:1035] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee0993578c7246e89854f4c838d4d946" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:43.236927 7584 mvcc.cc:205] Tried to move back new op lower bound from 7316066972602363904 to 7316066972565192704. Current Snapshot: MvccSnapshot[applied={T|T < 7316066972602363904}]
I20260808 00:32:43.237569 7585 mvcc.cc:205] Tried to move back new op lower bound from 7316066972602363904 to 7316066972565192704. Current Snapshot: MvccSnapshot[applied={T|T < 7316066972602363904}]
I20260808 00:32:43.239398 6847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:43.239498 6847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:43.240293 6847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 034bcaf5bb7846d1b3577f87dcae9dbd: Sending DeleteTablet for 3 replicas of tablet d76fb3d5c3184ef5b63a26832d670dce
I20260808 00:32:43.240702 7240 tablet_service.cc:1514] Processing DeleteTablet for tablet d76fb3d5c3184ef5b63a26832d670dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:43 UTC) from {username='slave'} at 127.0.0.1:54676
I20260808 00:32:43.240706 7438 tablet_service.cc:1514] Processing DeleteTablet for tablet d76fb3d5c3184ef5b63a26832d670dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:43 UTC) from {username='slave'} at 127.0.0.1:51756
I20260808 00:32:43.240739 7042 tablet_service.cc:1514] Processing DeleteTablet for tablet d76fb3d5c3184ef5b63a26832d670dce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:43 UTC) from {username='slave'} at 127.0.0.1:45042
I20260808 00:32:43.240986 7599 tablet_replica.cc:330] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: stopping tablet replica
I20260808 00:32:43.241106 7601 tablet_replica.cc:330] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34: stopping tablet replica
I20260808 00:32:43.241217 7601 raft_consensus.cc:2229] T d76fb3d5c3184ef5b63a26832d670dce P 503812825acd4fd7b7f53b55bb3dfc34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:43.241266 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 6955
I20260808 00:32:43.253223 7600 tablet_replica.cc:330] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: stopping tablet replica
I20260808 00:32:43.253459 7599 raft_consensus.cc:2229] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:43.253753 7600 raft_consensus.cc:2229] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:43.253927 7600 raft_consensus.cc:2258] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:43.254084 7599 raft_consensus.cc:2258] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:43.254781 7599 ts_tablet_manager.cc:1905] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:43.256686 7599 ts_tablet_manager.cc:1918] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:43.256686 7600 ts_tablet_manager.cc:1905] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:43.256882 7599 log.cc:1196] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/d76fb3d5c3184ef5b63a26832d670dce
I20260808 00:32:43.257288 7599 ts_tablet_manager.cc:1939] T d76fb3d5c3184ef5b63a26832d670dce P 6d015a7923404a419d252c5747b7a1c6: Deleting consensus metadata
I20260808 00:32:43.257835 6833 catalog_manager.cc:4841] TS 6d015a7923404a419d252c5747b7a1c6 (127.31.210.194:43351): tablet d76fb3d5c3184ef5b63a26832d670dce (table test-table [id=46a75c8df6e14c29a076f577700bc603]) successfully deleted
I20260808 00:32:43.258344 7600 ts_tablet_manager.cc:1918] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:43.258435 7600 log.cc:1196] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/d76fb3d5c3184ef5b63a26832d670dce
I20260808 00:32:43.258710 7600 ts_tablet_manager.cc:1939] T d76fb3d5c3184ef5b63a26832d670dce P ee0993578c7246e89854f4c838d4d946: Deleting consensus metadata
W20260808 00:32:43.258987 6834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:46813 (error 108)
I20260808 00:32:43.259284 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 7138
I20260808 00:32:43.266645 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.259389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46813 (local address 127.0.0.1:45048)
0808 00:32:43.266512 (+ 7123us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.266590 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:46813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7085}
W20260808 00:32:43.266815 6834 catalog_manager.cc:4584] TS 503812825acd4fd7b7f53b55bb3dfc34 (127.31.210.193:46813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d76fb3d5c3184ef5b63a26832d670dce: Network error: Client connection negotiation failed: client connection to 127.31.210.193:46813: connect: Connection refused (error 111)
I20260808 00:32:43.267946 6833 catalog_manager.cc:4841] TS ee0993578c7246e89854f4c838d4d946 (127.31.210.195:34189): tablet d76fb3d5c3184ef5b63a26832d670dce (table test-table [id=46a75c8df6e14c29a076f577700bc603]) successfully deleted
I20260808 00:32:43.274016 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 7349
I20260808 00:32:43.289067 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 6813
2026-08-08T00:32:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (671 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 00:32:43.313287 32587 mini_oidc.cc:152] Starting JWKS server
I20260808 00:32:43.314474 32587 webserver.cc:460] Webserver started at https://127.0.0.1:42623/ using document root <none> and password file <none>
I20260808 00:32:43.314615 32587 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:32:43.314805 32587 webserver.cc:460] Webserver started at http://127.0.0.1:42215/ using document root <none> and password file <none>
2026-08-08T00:32:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:43Z Disabled control of system clock
I20260808 00:32:43.323921 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:45493
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:35691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:45493
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42623/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/testchainca.cert with env {}
W20260808 00:32:43.407850 7610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:43.408077 7610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:43.408102 7610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:43.409296 7610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:43.409339 7610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:43.409353 7610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:43.409366 7610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:43.409379 7610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:32:43.409390 7610 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:32:43.409402 7610 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42623/jwks.json
I20260808 00:32:43.410930 7610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:45493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:45493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42623/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:43.411146 7610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:43.411325 7610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:43.416338 7619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.416348 7616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.416352 7626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:43.416697 7610 server_base.cc:1034] running on GCE node
I20260808 00:32:43.416935 7610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:43.417263 7610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:43.418418 7610 hybrid_clock.cc:648] HybridClock initialized: now 1786149163418396 us; error 39 us; skew 500 ppm
I20260808 00:32:43.419672 7610 webserver.cc:460] Webserver started at http://127.31.210.254:45371/ using document root <none> and password file <none>
I20260808 00:32:43.419886 7610 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:43.419927 7610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:43.420079 7610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:43.421170 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "735dcd8c9bb34c8bb1d2515983f75dbc"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "6afffb919292af9058d10b4a8bde9045"
server_key_iv: "ec9a2906d2372fbcbe1a029fadd1ce15"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.421480 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "735dcd8c9bb34c8bb1d2515983f75dbc"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "6afffb919292af9058d10b4a8bde9045"
server_key_iv: "ec9a2906d2372fbcbe1a029fadd1ce15"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.422843 7610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:32:43.423578 7633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.423766 7610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:43.423867 7610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "735dcd8c9bb34c8bb1d2515983f75dbc"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "6afffb919292af9058d10b4a8bde9045"
server_key_iv: "ec9a2906d2372fbcbe1a029fadd1ce15"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.423941 7610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:43.442471 7610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:43.442868 7610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:43.443058 7610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:43.447875 7610 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:45493
I20260808 00:32:43.447911 7702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:45493 every 8 connection(s)
I20260808 00:32:43.448352 7610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:43.448689 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 7610
I20260808 00:32:43.448834 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:43.449062 32587 external_mini_cluster.cc:1428] Setting key 40d5d1bbb8b885ba72fb2160a1f4ba6f
I20260808 00:32:43.449774 7704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:43.452595 7704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc: Bootstrap starting.
I20260808 00:32:43.453238 7704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:43.453531 7704 log.cc:824] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:43.454466 7704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc: No bootstrap required, opened a new log
I20260808 00:32:43.455152 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.449685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37756 (local address 127.31.210.254:45493)
0808 00:32:43.449952 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.449960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.449987 (+ 27us) server_negotiation.cc:407] Connection header received
0808 00:32:43.450460 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.450467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.450540 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.450707 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.450951 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.451921 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.452995 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.453508 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.453673 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:43.454405 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.454409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.454409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.454634 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.454670 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.454681 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.454736 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.454861 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.454930 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
I20260808 00:32:43.456305 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:32:43.457127 7704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45493 } }
I20260808 00:32:43.457338 7704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:43.457406 7704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735dcd8c9bb34c8bb1d2515983f75dbc, State: Initialized, Role: FOLLOWER
I20260808 00:32:43.457574 7704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45493 } }
I20260808 00:32:43.457680 7704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:43.457707 7704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:43.457744 7704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:43.458294 7704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45493 } }
I20260808 00:32:43.458398 7704 leader_election.cc:302] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 735dcd8c9bb34c8bb1d2515983f75dbc; no voters:
I20260808 00:32:43.458539 7704 leader_election.cc:288] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:43.458609 7715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:43.458765 7715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [term 1 LEADER]: Becoming Leader. State: Replica: 735dcd8c9bb34c8bb1d2515983f75dbc, State: Running, Role: LEADER
I20260808 00:32:43.458849 7704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:43.458856 7715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45493 } }
I20260808 00:32:43.459336 7716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45493 } } }
I20260808 00:32:43.459444 7716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:43.459657 7723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:43.460026 7717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 735dcd8c9bb34c8bb1d2515983f75dbc. Latest consensus state: current_term: 1 leader_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735dcd8c9bb34c8bb1d2515983f75dbc" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45493 } } }
I20260808 00:32:43.460171 7723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:43.460196 7717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:43.461789 7723 catalog_manager.cc:1347] Generated new cluster ID: 66bbf79354bd43d6b918677d6b18aa9d
I20260808 00:32:43.461862 7723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:43.472281 7723 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:43.473168 7723 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:43.475850 7723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 735dcd8c9bb34c8bb1d2515983f75dbc: Generated new TSK 0
I20260808 00:32:43.476102 7723 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 00:32:43.513577 7749 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 00:32:43.513734 7749 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 00:32:43.513832 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.490163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37762 (local address 127.31.210.254:45493)
0808 00:32:43.490210 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.490212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.490379 (+ 167us) server_negotiation.cc:407] Connection header received
0808 00:32:43.490497 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.490498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.490509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.490567 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:32:43.490739 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.491227 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.492369 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.492654 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.492714 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:43.493037 (+ 323us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:32:43.513531 (+ 20494us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 00:32:43.513734 (+ 203us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37762: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:32:43.513917 7707 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37762: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 00:32:43.514279 7747 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:43.490114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:45493 (local address 127.0.0.1:37762)
0808 00:32:43.490311 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.490324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.490403 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.490598 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.490599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.490601 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:32:43.490674 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:43.490677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.491302 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.491303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:43.492266 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:43.492270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.492747 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.492748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:43.492820 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:43.492845 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:32:43.513703 (+ 20858us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 00:32:43.513832 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.210.254:45493: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.210.254:45493: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 00:32:43.514870 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 7610
2026-08-08T00:32:43Z chronyd exiting
I20260808 00:32:43.557768 32587 test_util.cc:175] -----------------------------------------------
I20260808 00:32:43.557883 32587 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149138237870-32587-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (245 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 00:32:43.558744 32587 mini_oidc.cc:152] Starting JWKS server
I20260808 00:32:43.559511 32587 webserver.cc:460] Webserver started at https://127.0.0.1:42949/ using document root <none> and password file <none>
I20260808 00:32:43.559584 32587 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:32:43.559769 32587 webserver.cc:460] Webserver started at http://127.0.0.1:42649/ using document root <none> and password file <none>
2026-08-08T00:32:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:43Z Disabled control of system clock
I20260808 00:32:43.569079 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:45285
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:39705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:45285
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42949/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/testchainca.cert with env {}
W20260808 00:32:43.659097 7758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:43.659487 7758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:43.659577 7758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:43.661705 7758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:43.661784 7758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:43.661810 7758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:43.661836 7758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:43.661859 7758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:32:43.661883 7758 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:32:43.661914 7758 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42949/jwks.json
I20260808 00:32:43.664506 7758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:39705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:45285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:45285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42949/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:43.664783 7758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:43.665053 7758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:43.671295 7765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.671293 7766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:43.671597 7758 server_base.cc:1034] running on GCE node
W20260808 00:32:43.671662 7769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:43.671913 7758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:43.672230 7758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:43.673420 7758 hybrid_clock.cc:648] HybridClock initialized: now 1786149163673391 us; error 45 us; skew 500 ppm
I20260808 00:32:43.675168 7758 webserver.cc:460] Webserver started at http://127.31.210.254:42511/ using document root <none> and password file <none>
I20260808 00:32:43.675450 7758 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:43.675525 7758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:43.675670 7758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:43.677363 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "a7ce97c9a3de4ff6a401043bc817143d"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "589cb52e9c5b8797e37b53bde9b159fe"
server_key_iv: "ad11c6cecf65aecd5557c65ec7b491d1"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.677999 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "a7ce97c9a3de4ff6a401043bc817143d"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "589cb52e9c5b8797e37b53bde9b159fe"
server_key_iv: "ad11c6cecf65aecd5557c65ec7b491d1"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.679947 7758 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 00:32:43.680804 7778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.681028 7758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:43.681136 7758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "a7ce97c9a3de4ff6a401043bc817143d"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "589cb52e9c5b8797e37b53bde9b159fe"
server_key_iv: "ad11c6cecf65aecd5557c65ec7b491d1"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.681228 7758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:43.728909 7758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:43.729221 7758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:43.729334 7758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:43.733609 7758 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:45285
I20260808 00:32:43.733647 7852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:45285 every 8 connection(s)
I20260808 00:32:43.734016 7758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:43.734232 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 7758
I20260808 00:32:43.734347 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:43.734573 32587 external_mini_cluster.cc:1428] Setting key 72b69f04b671adbdc9517997c39b73d4
I20260808 00:32:43.734834 7854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:43.738463 7854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d: Bootstrap starting.
I20260808 00:32:43.739333 7854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:43.739670 7854 log.cc:824] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:43.740561 7854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d: No bootstrap required, opened a new log
I20260808 00:32:43.741093 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.735203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.31.210.254:45285)
0808 00:32:43.735454 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.735461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.735485 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:32:43.735913 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.735919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.735978 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.736130 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.736417 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.737824 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.738754 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.739457 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.739600 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:43.740349 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.740353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.740353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.740584 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.740641 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.740657 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.740721 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.740807 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.740888 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":124,"threads_started":1}
I20260808 00:32:43.742226 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:32:43.742884 7854 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ce97c9a3de4ff6a401043bc817143d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45285 } }
I20260808 00:32:43.743110 7854 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:43.743167 7854 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ce97c9a3de4ff6a401043bc817143d, State: Initialized, Role: FOLLOWER
I20260808 00:32:43.743304 7854 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ce97c9a3de4ff6a401043bc817143d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45285 } }
I20260808 00:32:43.743384 7854 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:43.743407 7854 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:43.743434 7854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:43.743924 7854 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ce97c9a3de4ff6a401043bc817143d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45285 } }
I20260808 00:32:43.744022 7854 leader_election.cc:302] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7ce97c9a3de4ff6a401043bc817143d; no voters:
I20260808 00:32:43.744146 7854 leader_election.cc:288] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:43.744239 7860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:43.744408 7860 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [term 1 LEADER]: Becoming Leader. State: Replica: a7ce97c9a3de4ff6a401043bc817143d, State: Running, Role: LEADER
I20260808 00:32:43.744426 7854 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:43.744534 7860 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ce97c9a3de4ff6a401043bc817143d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45285 } }
I20260808 00:32:43.744959 7861 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7ce97c9a3de4ff6a401043bc817143d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ce97c9a3de4ff6a401043bc817143d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45285 } } }
I20260808 00:32:43.745056 7861 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:43.745275 7869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:43.745296 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ce97c9a3de4ff6a401043bc817143d. Latest consensus state: current_term: 1 leader_uuid: "a7ce97c9a3de4ff6a401043bc817143d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ce97c9a3de4ff6a401043bc817143d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45285 } } }
I20260808 00:32:43.745491 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:43.745846 7869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:43.747490 7869 catalog_manager.cc:1347] Generated new cluster ID: 3d5692d7fa524e6c958ad91942bfaf9b
I20260808 00:32:43.747542 7869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:43.752290 7869 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:43.752863 7869 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:43.761528 7869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7ce97c9a3de4ff6a401043bc817143d: Generated new TSK 0
I20260808 00:32:43.761742 7869 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 00:32:43.789600 7895 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 00:32:43.789706 7895 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 00:32:43.790050 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.767606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.31.210.254:45285)
0808 00:32:43.767678 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.767680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.767802 (+ 122us) server_negotiation.cc:407] Connection header received
0808 00:32:43.767912 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.767914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.767928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.768008 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:32:43.768169 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.768754 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.769601 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.769822 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.769922 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:43.770140 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:32:43.789793 (+ 19653us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 00:32:43.789987 (+ 194us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38314: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 00:32:43.790148 7857 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38314: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 00:32:43.790206 7893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:43.767532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:45285 (local address 127.0.0.1:38314)
0808 00:32:43.767725 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:43.767736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:43.767822 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:43.768011 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:43.768012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:43.768014 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:32:43.768082 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:43.768085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.768871 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.768873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:43.769489 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:43.769491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.769926 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.769928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:43.769989 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:43.769995 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:32:43.789954 (+ 19959us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 00:32:43.790104 (+ 150us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.210.254:45285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42949/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":113,"threads_started":1}
I20260808 00:32:43.790824 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 7758
2026-08-08T00:32:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (276 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 00:32:43.835065 32587 mini_oidc.cc:152] Starting JWKS server
I20260808 00:32:43.835799 32587 webserver.cc:460] Webserver started at https://127.0.0.1:38643/ using document root <none> and password file <none>
I20260808 00:32:43.835875 32587 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:32:43.836045 32587 webserver.cc:460] Webserver started at http://127.0.0.1:43793/ using document root <none> and password file <none>
2026-08-08T00:32:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:43Z Disabled control of system clock
I20260808 00:32:43.845409 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:42235
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:34611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:42235
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38643/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 00:32:43.929644 7903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:43.929853 7903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:43.929875 7903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:43.931008 7903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:43.931046 7903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:43.931059 7903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:43.931072 7903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:43.931084 7903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:32:43.931099 7903 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:32:43.931111 7903 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38643/jwks.json
I20260808 00:32:43.932381 7903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:34611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:42235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:42235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38643/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:43.932675 7903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:43.932917 7903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:43.937932 7911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.937935 7914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:43.937981 7910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:43.938323 7903 server_base.cc:1034] running on GCE node
I20260808 00:32:43.938586 7903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:43.938930 7903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:43.940130 7903 hybrid_clock.cc:648] HybridClock initialized: now 1786149163940122 us; error 41 us; skew 500 ppm
I20260808 00:32:43.941435 7903 webserver.cc:460] Webserver started at http://127.31.210.254:37639/ using document root <none> and password file <none>
I20260808 00:32:43.941638 7903 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:43.941680 7903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:43.941774 7903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:43.942793 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "37f25d54f053436db0c15d4856d1299a"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "93b276743fa02d81462de3bd95fb19b7"
server_key_iv: "5873a4db51b9d6ab367f81cfb9954694"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.943235 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "37f25d54f053436db0c15d4856d1299a"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "93b276743fa02d81462de3bd95fb19b7"
server_key_iv: "5873a4db51b9d6ab367f81cfb9954694"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.944686 7903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:43.945436 7925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:43.945619 7903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:43.945730 7903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "37f25d54f053436db0c15d4856d1299a"
format_stamp: "Formatted at 2026-08-08 00:32:43 on dist-test-slave-44d2"
server_key: "93b276743fa02d81462de3bd95fb19b7"
server_key_iv: "5873a4db51b9d6ab367f81cfb9954694"
server_key_version: "encryptionkey@0"
I20260808 00:32:43.945808 7903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:43.967532 7903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:43.967900 7903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:43.968076 7903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:43.972393 7903 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:42235
I20260808 00:32:43.972430 8000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:42235 every 8 connection(s)
I20260808 00:32:43.972858 7903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:43.973582 8001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:43.976236 8001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a: Bootstrap starting.
I20260808 00:32:43.976868 8001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:43.977123 8001 log.cc:824] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:43.977780 8001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a: No bootstrap required, opened a new log
I20260808 00:32:43.979350 8001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37f25d54f053436db0c15d4856d1299a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42235 } }
I20260808 00:32:43.979488 8001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:43.979511 8001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37f25d54f053436db0c15d4856d1299a, State: Initialized, Role: FOLLOWER
I20260808 00:32:43.979660 8001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37f25d54f053436db0c15d4856d1299a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42235 } }
I20260808 00:32:43.979748 8001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:43.979771 8001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:43.979799 8001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:43.980233 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 7903
I20260808 00:32:43.980355 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:43.980321 8001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37f25d54f053436db0c15d4856d1299a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42235 } }
I20260808 00:32:43.980456 8001 leader_election.cc:302] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37f25d54f053436db0c15d4856d1299a; no voters:
I20260808 00:32:43.980628 32587 external_mini_cluster.cc:1428] Setting key b9985c5e158a07ab6c07c997bfd1339d
I20260808 00:32:43.980695 8001 leader_election.cc:288] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:43.980729 8004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:43.980885 8004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [term 1 LEADER]: Becoming Leader. State: Replica: 37f25d54f053436db0c15d4856d1299a, State: Running, Role: LEADER
I20260808 00:32:43.980988 8001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:43.981029 8004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37f25d54f053436db0c15d4856d1299a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42235 } }
I20260808 00:32:43.981370 8007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37f25d54f053436db0c15d4856d1299a. Latest consensus state: current_term: 1 leader_uuid: "37f25d54f053436db0c15d4856d1299a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37f25d54f053436db0c15d4856d1299a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42235 } } }
I20260808 00:32:43.981446 8007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:43.981701 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37f25d54f053436db0c15d4856d1299a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37f25d54f053436db0c15d4856d1299a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42235 } } }
I20260808 00:32:43.981777 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:43.981837 8011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:43.982429 8011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:43.984936 8011 catalog_manager.cc:1347] Generated new cluster ID: 356d3337ebab4c0b86c8a402fb48cf76
I20260808 00:32:43.984992 8011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:43.999820 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:43.981387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.31.210.254:42235)
0808 00:32:43.983192 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:43.983199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:43.983225 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:32:43.983314 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:43.983319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:43.983380 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:43.983534 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:43.983783 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.984739 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.997349 (+ 12610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:43.998051 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:43.998183 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:43.999015 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:43.999020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:43.999020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:43.999257 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:43.999299 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:43.999314 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:43.999377 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:43.999491 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 00:32:43.999578 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1739,"thread_start_us":266,"threads_started":1}
I20260808 00:32:44.000854 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:32:44.005075 8011 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:44.005765 8011 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:44.010398 8011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37f25d54f053436db0c15d4856d1299a: Generated new TSK 0
I20260808 00:32:44.010619 8011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:32:44.021140 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:44.017314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.31.210.254:42235)
0808 00:32:44.017378 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:44.017381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:44.017484 (+ 103us) server_negotiation.cc:407] Connection header received
0808 00:32:44.017572 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:44.017573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:44.017584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:44.017646 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:32:44.017807 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.018327 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.019482 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.019678 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.019728 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:44.019926 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:32:44.020805 (+ 879us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 00:32:44.020990 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47146: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 00:32:44.021214 8010 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47146: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 00:32:44.021250 8036 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:44.017243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:42235 (local address 127.0.0.1:47146)
0808 00:32:44.017414 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:44.017425 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:44.017498 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:44.017667 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:44.017668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:44.017670 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:32:44.017738 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:44.017741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.018414 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.018415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:44.019388 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:44.019391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.019757 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.019758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:44.019822 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:44.019828 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:32:44.020915 (+ 1087us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 00:32:44.021146 (+ 231us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.210.254:42235: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":117,"threads_started":1}
I20260808 00:32:44.021534 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 7903
E20260808 00:32:44.035480 8037 webserver.cc:533] Webserver: sslize failed:
E20260808 00:32:44.035564 8037 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T00:32:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (230 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T00:32:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:44Z Disabled control of system clock
I20260808 00:32:44.075203 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:40257
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:43937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:40257
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/testchainca.cert with env {}
W20260808 00:32:44.160285 8041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:44.160533 8041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:44.160559 8041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:44.161761 8041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:44.161806 8041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:44.161819 8041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:44.161832 8041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:44.161892 8041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:32:44.161908 8041 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:32:44.161933 8041 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 00:32:44.163504 8041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:40257
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:40257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:44.163791 8041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:44.164026 8041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:44.169164 8047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:44.169178 8048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:44.169188 8050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:44.169656 8041 server_base.cc:1034] running on GCE node
I20260808 00:32:44.169916 8041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:44.170243 8041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:44.171396 8041 hybrid_clock.cc:648] HybridClock initialized: now 1786149164171367 us; error 47 us; skew 500 ppm
I20260808 00:32:44.172776 8041 webserver.cc:460] Webserver started at http://127.31.210.254:38437/ using document root <none> and password file <none>
I20260808 00:32:44.173040 8041 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:44.173090 8041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:44.173261 8041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:44.174341 8041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "658724308a234e11b50cbcb13a6fa826"
format_stamp: "Formatted at 2026-08-08 00:32:44 on dist-test-slave-44d2"
server_key: "c68c274a67487f0ed2b29a4b1cd51d1c"
server_key_iv: "8d6ed59525d81b3e3254e08a96e1e0c7"
server_key_version: "encryptionkey@0"
I20260808 00:32:44.174726 8041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "658724308a234e11b50cbcb13a6fa826"
format_stamp: "Formatted at 2026-08-08 00:32:44 on dist-test-slave-44d2"
server_key: "c68c274a67487f0ed2b29a4b1cd51d1c"
server_key_iv: "8d6ed59525d81b3e3254e08a96e1e0c7"
server_key_version: "encryptionkey@0"
I20260808 00:32:44.176062 8041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:44.176872 8056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:44.177057 8041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:44.177165 8041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "658724308a234e11b50cbcb13a6fa826"
format_stamp: "Formatted at 2026-08-08 00:32:44 on dist-test-slave-44d2"
server_key: "c68c274a67487f0ed2b29a4b1cd51d1c"
server_key_iv: "8d6ed59525d81b3e3254e08a96e1e0c7"
server_key_version: "encryptionkey@0"
I20260808 00:32:44.177240 8041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:44.191644 8041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:44.191998 8041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:44.192164 8041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:44.196347 8041 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:40257
I20260808 00:32:44.196383 8138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:40257 every 8 connection(s)
I20260808 00:32:44.196812 8041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:44.197902 8139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:44.200053 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 8041
I20260808 00:32:44.200172 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:44.200357 32587 external_mini_cluster.cc:1428] Setting key eca60d604d625524f898b06136ff3736
I20260808 00:32:44.200366 8139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826: Bootstrap starting.
I20260808 00:32:44.201164 8139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:44.201525 8139 log.cc:824] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:44.202328 8139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826: No bootstrap required, opened a new log
I20260808 00:32:44.204758 8139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658724308a234e11b50cbcb13a6fa826" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40257 } }
I20260808 00:32:44.204941 8139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:44.204979 8139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658724308a234e11b50cbcb13a6fa826, State: Initialized, Role: FOLLOWER
I20260808 00:32:44.205097 8139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658724308a234e11b50cbcb13a6fa826" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40257 } }
I20260808 00:32:44.205184 8139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:44.205214 8139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:44.205260 8139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:44.206017 8139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658724308a234e11b50cbcb13a6fa826" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40257 } }
I20260808 00:32:44.206138 8139 leader_election.cc:302] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 658724308a234e11b50cbcb13a6fa826; no voters:
I20260808 00:32:44.206377 8139 leader_election.cc:288] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:44.206446 8145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:44.206594 8145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [term 1 LEADER]: Becoming Leader. State: Replica: 658724308a234e11b50cbcb13a6fa826, State: Running, Role: LEADER
I20260808 00:32:44.206663 8139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:44.206692 8145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658724308a234e11b50cbcb13a6fa826" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40257 } }
I20260808 00:32:44.207016 8145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 658724308a234e11b50cbcb13a6fa826. Latest consensus state: current_term: 1 leader_uuid: "658724308a234e11b50cbcb13a6fa826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658724308a234e11b50cbcb13a6fa826" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40257 } } }
I20260808 00:32:44.207104 8145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:44.207057 8149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "658724308a234e11b50cbcb13a6fa826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658724308a234e11b50cbcb13a6fa826" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40257 } } }
I20260808 00:32:44.207243 8149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:44.207471 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:44.201076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.31.210.254:40257)
0808 00:32:44.201338 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:44.201349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:44.201390 (+ 41us) server_negotiation.cc:407] Connection header received
0808 00:32:44.202003 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:44.202013 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:44.202095 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:44.202279 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:44.202575 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.204007 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.204881 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.205496 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.205628 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:44.206378 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:44.206383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:44.206383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:32:44.206618 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:44.206659 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:32:44.206674 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:44.206966 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:44.206999 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 00:32:44.207092 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260808 00:32:44.208020 8154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:44.208361 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:32:44.208750 8154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:44.211129 8154 catalog_manager.cc:1347] Generated new cluster ID: 6d06a194405b43799d6df14b85d1d4c4
I20260808 00:32:44.211186 8154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:44.229092 8154 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:44.229667 8154 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:44.234728 8154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 658724308a234e11b50cbcb13a6fa826: Generated new TSK 0
I20260808 00:32:44.234947 8154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:32:45.527025 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:44.247411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.31.210.254:40257)
0808 00:32:44.247479 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:44.247481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:44.247614 (+ 133us) server_negotiation.cc:407] Connection header received
0808 00:32:44.247696 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:44.247697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:44.247708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:44.247771 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:32:44.247924 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.248398 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.249596 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.249793 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.249844 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:44.250081 (+ 237us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:32:45.526719 (+1276638us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 00:32:45.526941 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48848: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 00:32:45.527115 8143 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48848: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 00:32:45.527208 8176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:44.247349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:40257 (local address 127.0.0.1:48848)
0808 00:32:44.247536 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:44.247549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:44.247623 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:44.247789 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:44.247790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:44.247792 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:32:44.247859 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:44.247862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.248516 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.248517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:44.249492 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:44.249495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:44.249888 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:44.249890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:44.249974 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:44.249980 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:32:45.526881 (+1276901us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 00:32:45.527065 (+ 184us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.210.254:40257: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
I20260808 00:32:45.527618 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 8041
2026-08-08T00:32:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1487 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (97 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (93 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:45 dist-test-slave-44d2 krb5kdc[8193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:45 dist-test-slave-44d2 krb5kdc[8193](info): set up 2 sockets
Aug 08 00:32:45 dist-test-slave-44d2 krb5kdc[8193](info): commencing operation
krb5kdc: starting...
W20260808 00:32:47.025156 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:47.161103 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:36779
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:36779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:47.247134 8210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:47.247398 8210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:47.247442 8210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:47.248718 8210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:47.248786 8210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:47.248816 8210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:47.248872 8210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:47.248907 8210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:47.250442 8210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:36779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:36779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:47.250742 8210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:47.251012 8210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:32:47.256162 8210 server_base.cc:1034] running on GCE node
W20260808 00:32:47.256145 8217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.256136 8220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.256145 8218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:47.256681 8210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:47.256947 8210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:47.258116 8210 hybrid_clock.cc:648] HybridClock initialized: now 1786149167258101 us; error 33 us; skew 500 ppm
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.260775 8210 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:32:47.261256 8210 webserver.cc:460] Webserver started at http://127.31.210.254:34271/ using document root <none> and password file <none>
I20260808 00:32:47.261533 8210 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:47.261581 8210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:47.261734 8210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:47.262815 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "69e8da91ff71408cb87842736a374b95"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "3ec84c3e74393340f99dfd7ef72a94e5"
server_key_iv: "daa1201d799e83a326b187947d101055"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.263185 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "69e8da91ff71408cb87842736a374b95"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "3ec84c3e74393340f99dfd7ef72a94e5"
server_key_iv: "daa1201d799e83a326b187947d101055"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.264626 8210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:47.265388 8228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.265573 8210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:47.265684 8210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "69e8da91ff71408cb87842736a374b95"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "3ec84c3e74393340f99dfd7ef72a94e5"
server_key_iv: "daa1201d799e83a326b187947d101055"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.265761 8210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:47.283290 8210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:47.285844 8210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:47.286029 8210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:47.290261 8210 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:36779
I20260808 00:32:47.290315 8305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:36779 every 8 connection(s)
I20260808 00:32:47.290709 8210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:47.291661 8306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:47.294026 8306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95: Bootstrap starting.
I20260808 00:32:47.294699 8306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:47.295018 8306 log.cc:824] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:47.295738 8306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95: No bootstrap required, opened a new log
I20260808 00:32:47.296700 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 8210
I20260808 00:32:47.296814 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:47.297003 32587 external_mini_cluster.cc:1428] Setting key 14e266145e13196ad3b7d754dd00becf
I20260808 00:32:47.297986 8306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e8da91ff71408cb87842736a374b95" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36779 } }
I20260808 00:32:47.298146 8306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:47.298169 8306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e8da91ff71408cb87842736a374b95, State: Initialized, Role: FOLLOWER
I20260808 00:32:47.298321 8306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e8da91ff71408cb87842736a374b95" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36779 } }
I20260808 00:32:47.298408 8306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:47.298452 8306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:47.298525 8306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:47.299397 8306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e8da91ff71408cb87842736a374b95" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36779 } }
I20260808 00:32:47.299525 8306 leader_election.cc:302] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69e8da91ff71408cb87842736a374b95; no voters:
I20260808 00:32:47.299721 8306 leader_election.cc:288] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:47.299757 8311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:47.299865 8311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [term 1 LEADER]: Becoming Leader. State: Replica: 69e8da91ff71408cb87842736a374b95, State: Running, Role: LEADER
I20260808 00:32:47.299948 8311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e8da91ff71408cb87842736a374b95" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36779 } }
I20260808 00:32:47.300009 8306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:47.300279 8312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69e8da91ff71408cb87842736a374b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e8da91ff71408cb87842736a374b95" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36779 } } }
I20260808 00:32:47.300390 8312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:47.300635 8319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:47.300670 8313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69e8da91ff71408cb87842736a374b95. Latest consensus state: current_term: 1 leader_uuid: "69e8da91ff71408cb87842736a374b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e8da91ff71408cb87842736a374b95" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36779 } } }
I20260808 00:32:47.300873 8313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:47.301218 8319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:47.303009 8319 catalog_manager.cc:1347] Generated new cluster ID: 8f0e31f1af1f4d0384238b0c520b1c96
I20260808 00:32:47.303067 8319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:47.312072 8319 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:47.312901 8319 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:47.314491 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.297619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.31.210.254:36779)
0808 00:32:47.297820 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.297829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.297855 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:32:47.298051 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.298059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.298118 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.298282 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.299036 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.300289 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.301540 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.302155 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.302271 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.310378 (+ 8107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:47.310384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:47.310385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:47.310406 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:47.312685 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.313149 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.313154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.313154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.313276 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.313508 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.313509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.313510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.313798 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:47.313927 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:47.314088 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 00:32:47.314243 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260808 00:32:47.318735 8319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69e8da91ff71408cb87842736a374b95: Generated new TSK 0
I20260808 00:32:47.318933 8319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:32:47.352439 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:36779
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:47.440644 8346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:47.440869 8346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:47.440896 8346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:47.442090 8346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:47.442137 8346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:47.442198 8346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:47.443708 8346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:36779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:47.444003 8346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:47.444259 8346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:47.449790 8353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.449807 8357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.449791 8352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:47.450031 8346 server_base.cc:1034] running on GCE node
I20260808 00:32:47.450358 8346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:47.450614 8346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:47.451778 8346 hybrid_clock.cc:648] HybridClock initialized: now 1786149167451756 us; error 38 us; skew 500 ppm
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.454391 8346 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:32:47.454857 8346 webserver.cc:460] Webserver started at http://127.31.210.193:34349/ using document root <none> and password file <none>
I20260808 00:32:47.455128 8346 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:47.455204 8346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:47.455353 8346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:47.456367 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "71c13ec4ae814da281d447eacf84a2e6"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "f67e511fb958b47fdfd09cfc2e47614f"
server_key_iv: "25153cb4ad7d9e8921a5efde146a62c2"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.456858 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "71c13ec4ae814da281d447eacf84a2e6"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "f67e511fb958b47fdfd09cfc2e47614f"
server_key_iv: "25153cb4ad7d9e8921a5efde146a62c2"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.458245 8346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:47.459035 8368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.459208 8346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:47.459311 8346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "71c13ec4ae814da281d447eacf84a2e6"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "f67e511fb958b47fdfd09cfc2e47614f"
server_key_iv: "25153cb4ad7d9e8921a5efde146a62c2"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.459390 8346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:47.477746 8346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:47.479712 8346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:47.479892 8346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:47.480171 8346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:47.480549 8346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:47.480608 8346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.480645 8346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:47.480674 8346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.486837 8346 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:38591
I20260808 00:32:47.486889 8529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:38591 every 8 connection(s)
I20260808 00:32:47.487228 8346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:47.488754 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 8346
I20260808 00:32:47.488845 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:47.489071 32587 external_mini_cluster.cc:1428] Setting key dc547b3593729e55f5fab6d6046d4b65
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:47.498699 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.488007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:45919 (local address 127.31.210.254:36779)
0808 00:32:47.488085 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.488088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.488319 (+ 231us) server_negotiation.cc:407] Connection header received
0808 00:32:47.489598 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.489599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.489611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.489663 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.490437 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.490942 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.491985 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.492192 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.492269 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.494820 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:47.494822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:47.494822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:47.494832 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:47.496838 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.497590 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.497592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.497592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.497660 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.497970 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.497971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.497972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.498111 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:47.498196 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:47.498495 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 00:32:47.498599 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:32:47.498919 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.487957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:36779 (local address 127.31.210.193:45919)
0808 00:32:47.488185 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:47.488228 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:47.489488 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:47.489753 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:47.489761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:47.490123 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:47.490326 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:47.490331 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.491049 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.491051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:47.491871 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:47.491873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.492278 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.492279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:47.492421 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.493121 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:47.493130 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:47.494715 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:47.497111 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:47.497119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:47.497122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:47.497496 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:47.497745 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:47.497746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:47.497747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:47.497877 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:47.498185 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:47.498193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:47.498342 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 00:32:47.498467 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 00:32:47.498641 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":120,"threads_started":1}
I20260808 00:32:47.499679 8530 heartbeater.cc:347] Connected to a master server at 127.31.210.254:36779
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
I20260808 00:32:47.499801 8530 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:47.500106 8530 heartbeater.cc:511] Master 127.31.210.254:36779 requested a full tablet report, sending...
I20260808 00:32:47.500788 8257 ts_manager.cc:194] Registered new tserver with Master: 71c13ec4ae814da281d447eacf84a2e6 (127.31.210.193:38591)
I20260808 00:32:47.501454 8257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:45919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:32:47.519554 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:36779
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:47.605240 8536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:47.605444 8536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:47.605470 8536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:47.606594 8536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:47.606633 8536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:47.606660 8536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:47.608071 8536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:36779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:47.608258 8536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:47.608440 8536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:32:47.613765 8536 server_base.cc:1034] running on GCE node
W20260808 00:32:47.613730 8545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.613730 8550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.613725 8547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:47.614234 8536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:47.614506 8536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:47.615681 8536 hybrid_clock.cc:648] HybridClock initialized: now 1786149167615668 us; error 46 us; skew 500 ppm
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.618398 8536 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:32:47.618798 8536 webserver.cc:460] Webserver started at http://127.31.210.194:44235/ using document root <none> and password file <none>
I20260808 00:32:47.619065 8536 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:47.619124 8536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:47.619257 8536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:47.620335 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "50150bc1dbbc4a9fa439cd7f1b418651"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "96f448ffdb4e0e7b7ed9c085f8be3f7a"
server_key_iv: "90c51b99efc3f4fd9e1df568eb6a9b51"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.620750 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "50150bc1dbbc4a9fa439cd7f1b418651"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "96f448ffdb4e0e7b7ed9c085f8be3f7a"
server_key_iv: "90c51b99efc3f4fd9e1df568eb6a9b51"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.622184 8536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:47.623025 8560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.623273 8536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:47.623351 8536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "50150bc1dbbc4a9fa439cd7f1b418651"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "96f448ffdb4e0e7b7ed9c085f8be3f7a"
server_key_iv: "90c51b99efc3f4fd9e1df568eb6a9b51"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.623442 8536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:47.642817 8536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:47.645239 8536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:47.645417 8536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:47.645680 8536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:47.646015 8536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:47.646073 8536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.646107 8536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:47.646138 8536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.651981 8536 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:41245
I20260808 00:32:47.652052 8723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:41245 every 8 connection(s)
I20260808 00:32:47.652407 8536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:47.655778 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 8536
I20260808 00:32:47.655900 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:47.656105 32587 external_mini_cluster.cc:1428] Setting key bcde62d5f164245154f3eaafd2941550
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:47.663568 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.653263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:34047 (local address 127.31.210.254:36779)
0808 00:32:47.653331 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.653335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.653563 (+ 228us) server_negotiation.cc:407] Connection header received
0808 00:32:47.654510 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.654511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.654522 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.654580 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.655198 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.655690 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.657148 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.657345 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.657422 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.660208 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:47.660210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:47.660210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:47.660220 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:47.661995 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.662495 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.662497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.662497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.662566 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.662812 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.662813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.662813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.662983 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:47.663068 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:47.663332 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 00:32:47.663424 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:32:47.663638 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.653256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:36779 (local address 127.31.210.194:34047)
0808 00:32:47.653425 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:47.653462 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:47.654410 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:47.654658 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:47.654661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:47.654914 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:47.655126 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:47.655129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.655791 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.655793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:47.657040 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:47.657043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.657446 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.657448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:47.657595 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.658162 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:47.658172 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:47.660095 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:47.662183 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:47.662190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:47.662192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:47.662412 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:47.662642 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:47.662642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:47.662643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:47.662743 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:47.663057 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:47.663062 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:47.663176 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 00:32:47.663305 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 00:32:47.663433 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260808 00:32:47.664180 8725 heartbeater.cc:347] Connected to a master server at 127.31.210.254:36779
I20260808 00:32:47.664299 8725 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:47.664541 8725 heartbeater.cc:511] Master 127.31.210.254:36779 requested a full tablet report, sending...
I20260808 00:32:47.664958 8257 ts_manager.cc:194] Registered new tserver with Master: 50150bc1dbbc4a9fa439cd7f1b418651 (127.31.210.194:41245)
I20260808 00:32:47.665350 8257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:34047
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:32:47.687350 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:36779
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:47.771371 8730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:47.771582 8730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:47.771607 8730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:47.772800 8730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:47.772843 8730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:47.772869 8730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:47.774340 8730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:36779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:47.774597 8730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:47.774840 8730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:47.780217 8738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.780289 8736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:47.780222 8740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:47.780514 8730 server_base.cc:1034] running on GCE node
I20260808 00:32:47.780756 8730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:47.780962 8730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:47.782104 8730 hybrid_clock.cc:648] HybridClock initialized: now 1786149167782080 us; error 33 us; skew 500 ppm
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.784673 8730 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:32:47.785130 8730 webserver.cc:460] Webserver started at http://127.31.210.195:45303/ using document root <none> and password file <none>
I20260808 00:32:47.785341 8730 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:47.785380 8730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:47.785481 8730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:47.786491 8730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "033b046aeb7448a68c476f5fc9aa7451"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "81fc0ba6a797d4766f02ad05b4987b47"
server_key_iv: "3f8048288492d2b1ee8f1bf7b3369600"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.787122 8730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "033b046aeb7448a68c476f5fc9aa7451"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "81fc0ba6a797d4766f02ad05b4987b47"
server_key_iv: "3f8048288492d2b1ee8f1bf7b3369600"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.788784 8730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:47.789512 8748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.789686 8730 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:47.789789 8730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "033b046aeb7448a68c476f5fc9aa7451"
format_stamp: "Formatted at 2026-08-08 00:32:47 on dist-test-slave-44d2"
server_key: "81fc0ba6a797d4766f02ad05b4987b47"
server_key_iv: "3f8048288492d2b1ee8f1bf7b3369600"
server_key_version: "encryptionkey@0"
I20260808 00:32:47.789875 8730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:47.814503 8730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:47.816880 8730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:47.817061 8730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:47.817344 8730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:47.817675 8730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:47.817731 8730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.817776 8730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:47.817803 8730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:47.824304 8730 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:39579
I20260808 00:32:47.824349 8918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:39579 every 8 connection(s)
I20260808 00:32:47.824750 8730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:32:47.833242 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 8730
I20260808 00:32:47.833351 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:47.833549 32587 external_mini_cluster.cc:1428] Setting key abd6218c8dbdfe5c4528872f9eb2516d
I20260808 00:32:47.834638 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.825539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:34491 (local address 127.31.210.254:36779)
0808 00:32:47.825612 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.825614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.825807 (+ 193us) server_negotiation.cc:407] Connection header received
0808 00:32:47.826799 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.826800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.826812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.826878 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.827514 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.828037 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.829115 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.829314 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.829381 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.831605 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:47.831606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:47.831606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:47.831613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:47.833008 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.833432 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.833433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.833434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.833506 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.833850 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.833851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.833851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.834009 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:47.834081 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:47.834433 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 00:32:47.834524 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:32:47.834786 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.825512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:36779 (local address 127.31.210.195:34491)
0808 00:32:47.825671 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:47.825715 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:47.826703 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:47.826937 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:47.826940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:47.827193 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:47.827433 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:47.827438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.828121 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.828122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:47.829030 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:47.829033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.829398 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.829399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:47.829517 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.830086 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:47.830094 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:47.831522 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:47.833087 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:47.833093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:47.833095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:47.833319 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:47.833582 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:47.833583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:47.833584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:47.833769 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:47.834086 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:47.834093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:47.834263 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 00:32:47.834383 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 00:32:47.834588 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260808 00:32:47.835235 8919 heartbeater.cc:347] Connected to a master server at 127.31.210.254:36779
I20260808 00:32:47.835352 8919 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:47.835531 8919 heartbeater.cc:511] Master 127.31.210.254:36779 requested a full tablet report, sending...
I20260808 00:32:47.835924 8257 ts_manager.cc:194] Registered new tserver with Master: 033b046aeb7448a68c476f5fc9aa7451 (127.31.210.195:39579)
I20260808 00:32:47.836351 8257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:34491
I20260808 00:32:47.845271 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/test-admin.keytab.
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.865649 32587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 00:32:47.867610 8926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:47.866721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.31.210.193:38591)
0808 00:32:47.866900 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.866906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.866939 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:32:47.866990 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.866992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.867006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.867086 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.867492 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51138: BlockingRecv error: recv got EOF from 127.0.0.1:51138 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
W20260808 00:32:47.867795 32587 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5cc' of type FILE
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.869040 32587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.870164 32587 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:47.878095 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.870706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.31.210.193:38591)
0808 00:32:47.870763 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.870764 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.870797 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:32:47.870856 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.870856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.870867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.870917 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.871285 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.872020 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.872840 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.873181 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.873238 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.875390 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:47.875399 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:47.875400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:47.875406 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:47.877090 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.877407 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.877410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.877410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.877466 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.877631 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.877632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.877632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.877762 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:47.877865 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:47.877932 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 00:32:47.877993 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 00:32:47.880023 8926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:47.879439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.31.210.193:38591)
0808 00:32:47.879493 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.879494 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.879530 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:32:47.879594 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.879595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.879605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.879650 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.879927 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51152: BlockingRecv error: recv got EOF from 127.0.0.1:51152 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:32:47.880051 32587 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5cc' of type FILE
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.881156 32587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.882004 32587 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:47.888732 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.882573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.31.210.193:38591)
0808 00:32:47.882627 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.882629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.882658 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:32:47.882707 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.882708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.882716 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.882761 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.883084 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.883551 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.884199 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.884385 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.884434 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.886354 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:47.886355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:47.886355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:47.886361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:47.887750 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.888030 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.888030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.888031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.888081 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.888244 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.888245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.888245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.888323 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:47.888393 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:47.888611 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 00:32:47.888673 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 00:32:47.890285 8926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:47.889761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.31.210.193:38591)
0808 00:32:47.889818 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.889819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.889831 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:47.889909 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.889910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.889918 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.889963 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.890240 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51172: BlockingRecv error: recv got EOF from 127.0.0.1:51172 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 00:32:47.890340 32587 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149138237870-32587-0/krb5kdc/krb5cc' of type FILE
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.891258 32587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:47.892107 32587 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:32:47.899336 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:47.892733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.31.210.193:38591)
0808 00:32:47.892781 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:47.892783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:47.892875 (+ 92us) server_negotiation.cc:407] Connection header received
0808 00:32:47.892989 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:47.892990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:47.892998 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:47.893044 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:47.893400 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.894046 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.894726 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:47.894978 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:47.895048 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:47.897164 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:47.897166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:47.897166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:47.897172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:47.898403 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.898702 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.898703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.898703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.898758 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:47.898981 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:47.898982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:47.898982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:47.899071 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:47.899142 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:47.899224 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:32:47.899281 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:32:47.900038 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 8346
I20260808 00:32:47.913095 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 8536
I20260808 00:32:47.922772 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 8730
I20260808 00:32:47.933279 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 8210
2026-08-08T00:32:47Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2210 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8942](info): set up 2 sockets
Aug 08 00:32:47 dist-test-slave-44d2 krb5kdc[8942](info): commencing operation
krb5kdc: starting...
W20260808 00:32:49.256553 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:49 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab.
Entry for principal oryx/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab.
I20260808 00:32:49.392052 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:34983
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab
--principal=oryx/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:34983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:49.477532 8964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:49.477778 8964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:49.477816 8964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:49.478969 8964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:49.479045 8964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:49.479068 8964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:49.479089 8964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:49.479120 8964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:49.480672 8964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:34983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab
--principal=oryx/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:34983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:49.480957 8964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:49.481196 8964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:49.486244 8970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:49.486263 8971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:49.486280 8975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:49.486801 8964 server_base.cc:1034] running on GCE node
I20260808 00:32:49.487146 8964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:49.487516 8964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:49.488684 8964 hybrid_clock.cc:648] HybridClock initialized: now 1786149169488664 us; error 36 us; skew 500 ppm
Aug 08 00:32:49 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:49.491287 8964 init.cc:377] Logged in from keytab as oryx/127.31.210.254@KRBTEST.COM (short username oryx)
I20260808 00:32:49.491802 8964 webserver.cc:460] Webserver started at http://127.31.210.254:36339/ using document root <none> and password file <none>
I20260808 00:32:49.492041 8964 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:49.492087 8964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:49.492264 8964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:49.493417 8964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "66554fb34779eb2baa5904a16b595c30"
server_key_iv: "ece38f65403b82b3c29ad7021b8a0953"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.493793 8964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "66554fb34779eb2baa5904a16b595c30"
server_key_iv: "ece38f65403b82b3c29ad7021b8a0953"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.495144 8964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:32:49.495867 8985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:49.496093 8964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:49.496182 8964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "66554fb34779eb2baa5904a16b595c30"
server_key_iv: "ece38f65403b82b3c29ad7021b8a0953"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.496236 8964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:49.524566 8964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:49.527091 8964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:49.527213 8964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:49.531213 8964 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:34983
I20260808 00:32:49.531273 9058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:34983 every 8 connection(s)
I20260808 00:32:49.531664 8964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:49.532908 9059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:49.535318 9059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: Bootstrap starting.
I20260808 00:32:49.536029 9059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:49.536343 9059 log.cc:824] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:49.537158 9059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: No bootstrap required, opened a new log
I20260808 00:32:49.538419 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 8964
I20260808 00:32:49.538524 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:49.538748 32587 external_mini_cluster.cc:1428] Setting key 4c7f65996d53c10180732e8b4173761a
I20260808 00:32:49.539050 9059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } }
I20260808 00:32:49.539224 9059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:49.539261 9059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3ba4458c064fc6bc5369f6b30a6ef6, State: Initialized, Role: FOLLOWER
I20260808 00:32:49.539388 9059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } }
I20260808 00:32:49.539505 9059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:49.539572 9059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:49.539629 9059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:49.540663 9059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } }
I20260808 00:32:49.540789 9059 leader_election.cc:302] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c3ba4458c064fc6bc5369f6b30a6ef6; no voters:
I20260808 00:32:49.540987 9059 leader_election.cc:288] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:49.541030 9065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:49.541158 9065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3ba4458c064fc6bc5369f6b30a6ef6, State: Running, Role: LEADER
I20260808 00:32:49.541266 9059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:49.541242 9065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } }
I20260808 00:32:49.541612 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3ba4458c064fc6bc5369f6b30a6ef6. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } }
I20260808 00:32:49.541694 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:49.541858 9068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } }
I20260808 00:32:49.541965 9068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:49.542115 9074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:49.542681 9074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:49.545132 9074 catalog_manager.cc:1347] Generated new cluster ID: 74dd0d7715f042418d187f293141dee1
I20260808 00:32:49.545199 9074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:49 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:49.549371 9074 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:49.550060 9074 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:49.550798 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:49.539325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.31.210.254:34983)
0808 00:32:49.539594 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:49.539605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:49.539642 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:32:49.539810 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:49.539816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:49.539862 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:49.540007 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:49.540718 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.542106 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.543323 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.543750 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.543875 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:49.546752 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:49.546756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:49.546757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:49.546769 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:49.548822 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:49.549262 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:49.549265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:49.549266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:49.549380 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:49.549596 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:49.549597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:49.549597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:49.549845 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:49.550008 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:49.550443 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 00:32:49.550587 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260808 00:32:49.552470 9074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: Generated new TSK 0
I20260808 00:32:49.552707 9074 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab.
Entry for principal oryx/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab.
I20260808 00:32:49.587852 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:34983
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab
--principal=oryx/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:49.672216 9096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:49.672420 9096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:49.672442 9096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:49.673733 9096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:49.673779 9096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:49.673804 9096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:49.675236 9096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab
--principal=oryx/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:34983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:49.675506 9096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:49.675743 9096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:49.681098 9105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:49.681206 9107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:49.681159 9102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:49.681277 9096 server_base.cc:1034] running on GCE node
I20260808 00:32:49.681468 9096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:49.681697 9096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:49.682868 9096 hybrid_clock.cc:648] HybridClock initialized: now 1786149169682860 us; error 38 us; skew 500 ppm
Aug 08 00:32:49 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:49.685637 9096 init.cc:377] Logged in from keytab as oryx/127.31.210.193@KRBTEST.COM (short username oryx)
I20260808 00:32:49.686079 9096 webserver.cc:460] Webserver started at http://127.31.210.193:40747/ using document root <none> and password file <none>
I20260808 00:32:49.686307 9096 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:49.686378 9096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:49.686537 9096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:49.687595 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "a690268c10b748f7995fed5d1bd9a483"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "236c1bebed27a98846484d8958e1b8d6"
server_key_iv: "5ff23e09df0cbd631ae032f61a5e3975"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.687984 9096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "a690268c10b748f7995fed5d1bd9a483"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "236c1bebed27a98846484d8958e1b8d6"
server_key_iv: "5ff23e09df0cbd631ae032f61a5e3975"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.689450 9096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:49.690220 9119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:49.690392 9096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:49.690521 9096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "a690268c10b748f7995fed5d1bd9a483"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "236c1bebed27a98846484d8958e1b8d6"
server_key_iv: "5ff23e09df0cbd631ae032f61a5e3975"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.690604 9096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:49.701583 9096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:49.703922 9096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:49.704097 9096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:49.704329 9096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:49.704694 9096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:49.704752 9096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:49.704804 9096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:49.704826 9096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:49.710846 9096 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:35275
I20260808 00:32:49.710899 9279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:35275 every 8 connection(s)
I20260808 00:32:49.711231 9096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:49.713639 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 9096
I20260808 00:32:49.713761 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:49.713974 32587 external_mini_cluster.cc:1428] Setting key 094631c1c70d83a26c6267a372cb92fc
Aug 08 00:32:49 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:49.721992 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:49.711885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:41463)
0808 00:32:49.712061 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:49.712105 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:49.712707 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:49.712925 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:49.712928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:49.713617 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:49.713928 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:49.713935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.714822 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.714823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:49.715683 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:49.715685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.716113 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.716114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:49.716295 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:49.716841 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:49.716853 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:49.718488 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:49.720534 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:49.720549 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:49.720550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:49.720775 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:49.721007 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:49.721007 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:49.721007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:49.721132 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:49.721467 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:49.721473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:49.721591 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 00:32:49.721689 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 00:32:49.721818 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260808 00:32:49.721989 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:49.711952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:41463 (local address 127.31.210.254:34983)
0808 00:32:49.712009 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:49.712015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:49.712182 (+ 167us) server_negotiation.cc:407] Connection header received
0808 00:32:49.712799 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:49.712800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:49.712810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:49.712871 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:49.714040 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.714723 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.715795 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.716009 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.716098 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:49.718580 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:49.718582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:49.718582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:49.718590 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:49.720378 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:49.720854 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:49.720855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:49.720856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:49.720929 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:49.721215 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:49.721217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:49.721217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:49.721385 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:49.721467 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:49.721777 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 00:32:49.721879 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:32:49.722596 9282 heartbeater.cc:347] Connected to a master server at 127.31.210.254:34983
I20260808 00:32:49.722694 9282 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:49.722899 9282 heartbeater.cc:511] Master 127.31.210.254:34983 requested a full tablet report, sending...
I20260808 00:32:49.723483 9008 ts_manager.cc:194] Registered new tserver with Master: a690268c10b748f7995fed5d1bd9a483 (127.31.210.193:35275)
I20260808 00:32:49.724100 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:41463
WARNING: no policy specified for oryx/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab.
Entry for principal oryx/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab.
I20260808 00:32:49.743575 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:34983
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab
--principal=oryx/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:49.829545 9288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:49.829805 9288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:49.829846 9288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:49.831032 9288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:49.831106 9288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:49.831153 9288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:49.832690 9288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab
--principal=oryx/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:34983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:49.832963 9288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:49.833215 9288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:49.838747 9303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:49.838778 9296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:49.838888 9288 server_base.cc:1034] running on GCE node
W20260808 00:32:49.839076 9299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:49.839320 9288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:49.839546 9288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:49.840687 9288 hybrid_clock.cc:648] HybridClock initialized: now 1786149169840676 us; error 28 us; skew 500 ppm
Aug 08 00:32:49 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:49.843627 9288 init.cc:377] Logged in from keytab as oryx/127.31.210.194@KRBTEST.COM (short username oryx)
I20260808 00:32:49.844172 9288 webserver.cc:460] Webserver started at http://127.31.210.194:38097/ using document root <none> and password file <none>
I20260808 00:32:49.844427 9288 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:49.844522 9288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:49.844712 9288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:49.845825 9288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "735b28ad727d446189a4e674982d1a1e"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "eb1b406f09299d02a3428611e77d8f59"
server_key_iv: "a488450be0ab1a2dedda54aaab3cdb06"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.846232 9288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "735b28ad727d446189a4e674982d1a1e"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "eb1b406f09299d02a3428611e77d8f59"
server_key_iv: "a488450be0ab1a2dedda54aaab3cdb06"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.847692 9288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:49.848640 9312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:49.848839 9288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:49.849020 9288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "735b28ad727d446189a4e674982d1a1e"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "eb1b406f09299d02a3428611e77d8f59"
server_key_iv: "a488450be0ab1a2dedda54aaab3cdb06"
server_key_version: "encryptionkey@0"
I20260808 00:32:49.849103 9288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:49.864449 9288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:49.866570 9288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:49.866753 9288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:49.867022 9288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:49.867393 9288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:49.867465 9288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:49.867503 9288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:49.867537 9288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:49.874578 9288 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:43181
I20260808 00:32:49.874648 9462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:43181 every 8 connection(s)
I20260808 00:32:49.875049 9288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:49.879832 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 9288
I20260808 00:32:49.879967 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:49.880189 32587 external_mini_cluster.cc:1428] Setting key c1316a452303b7288968ac3bcd57a573
Aug 08 00:32:49 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:49.886319 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:49.876257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:40049 (local address 127.31.210.254:34983)
0808 00:32:49.876305 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:49.876308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:49.876320 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:49.876875 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:49.876876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:49.876890 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:49.876968 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:49.878076 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.878578 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.879658 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.880028 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.880124 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:49.883081 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:49.883084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:49.883084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:49.883093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:49.884864 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:49.885306 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:49.885308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:49.885308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:49.885375 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:49.885663 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:49.885664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:49.885664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:49.885786 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:49.885860 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:49.886138 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 00:32:49.886225 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:32:49.886664 9467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:49.875927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:40049)
0808 00:32:49.876137 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:49.876179 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:49.876772 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:49.877022 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:49.877025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:49.877747 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:49.877956 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:49.877960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.878673 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.878674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:49.879525 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:49.879528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:49.880127 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:49.880129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:49.880243 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:49.881161 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:49.881173 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:49.882971 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:49.884947 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:49.884954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:49.884956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:49.885215 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:49.885453 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:49.885454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:49.885455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:49.885582 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:49.885862 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:49.885871 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:49.886004 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 00:32:49.886115 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 00:32:49.886277 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260808 00:32:49.887238 9463 heartbeater.cc:347] Connected to a master server at 127.31.210.254:34983
I20260808 00:32:49.887351 9463 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:49.887571 9463 heartbeater.cc:511] Master 127.31.210.254:34983 requested a full tablet report, sending...
I20260808 00:32:49.888039 9008 ts_manager.cc:194] Registered new tserver with Master: 735b28ad727d446189a4e674982d1a1e (127.31.210.194:43181)
I20260808 00:32:49.888458 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:40049
WARNING: no policy specified for oryx/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab.
Entry for principal oryx/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab.
I20260808 00:32:49.911660 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:34983
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab
--principal=oryx/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:49.997423 9473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:49.997648 9473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:49.997673 9473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:49.998996 9473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:49.999063 9473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:49.999094 9473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:50.000906 9473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab
--principal=oryx/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:34983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:50.001217 9473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:50.001490 9473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:50.007248 9484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.007409 9479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.007555 9481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.007767 9473 server_base.cc:1034] running on GCE node
I20260808 00:32:50.008046 9473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:50.008329 9473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:50.009505 9473 hybrid_clock.cc:648] HybridClock initialized: now 1786149170009503 us; error 45 us; skew 500 ppm
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:50.012401 9473 init.cc:377] Logged in from keytab as oryx/127.31.210.195@KRBTEST.COM (short username oryx)
I20260808 00:32:50.012982 9473 webserver.cc:460] Webserver started at http://127.31.210.195:32841/ using document root <none> and password file <none>
I20260808 00:32:50.013237 9473 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.013309 9473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.013465 9473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:50.014592 9473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "fa68f19a095045d7a18ec4bae119ba01"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "b89255722bfe8d2414d363058066ed0a"
server_key_iv: "c3767ff91b744a6365877dfc48ee70ee"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.014983 9473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "fa68f19a095045d7a18ec4bae119ba01"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "b89255722bfe8d2414d363058066ed0a"
server_key_iv: "c3767ff91b744a6365877dfc48ee70ee"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.016426 9473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:50.017349 9491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.017570 9473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:50.017680 9473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "fa68f19a095045d7a18ec4bae119ba01"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "b89255722bfe8d2414d363058066ed0a"
server_key_iv: "c3767ff91b744a6365877dfc48ee70ee"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.017766 9473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.030994 9473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:50.033609 9473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:50.033792 9473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:50.034051 9473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:50.034379 9473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:50.034435 9473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.034472 9473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:50.034519 9473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.041138 9473 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:45575
I20260808 00:32:50.041183 9647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:45575 every 8 connection(s)
I20260808 00:32:50.041531 9473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:50.047802 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 9473
I20260808 00:32:50.047892 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:50.048084 32587 external_mini_cluster.cc:1428] Setting key 92b87f5801d4a70e3ef9492faa4cc720
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:50.051906 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.042419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:34589 (local address 127.31.210.254:34983)
0808 00:32:50.042576 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.042579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.042840 (+ 261us) server_negotiation.cc:407] Connection header received
0808 00:32:50.043540 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.043542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.043553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.043618 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.044634 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.045155 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.046187 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.046370 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.046422 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.048947 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.048948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.048948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.048954 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.050317 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.050839 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.050841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.050841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.050908 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.051227 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.051228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.051229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.051355 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.051431 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.051713 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.051814 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 00:32:50.052141 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.042378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:34589)
0808 00:32:50.042648 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.042697 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.043424 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.043676 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.043679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.044277 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.044536 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.044543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.045248 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.045250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.046111 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.046113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.046443 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.046443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.046550 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.047113 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.047122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.048866 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.050390 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.050397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.050399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.050635 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.051043 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.051044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.051044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.051153 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.051432 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.051440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.051580 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.051691 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.051855 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":136,"threads_started":1}
I20260808 00:32:50.052796 9650 heartbeater.cc:347] Connected to a master server at 127.31.210.254:34983
I20260808 00:32:50.053078 9650 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:50.053364 9650 heartbeater.cc:511] Master 127.31.210.254:34983 requested a full tablet report, sending...
I20260808 00:32:50.053901 9008 ts_manager.cc:194] Registered new tserver with Master: fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575)
I20260808 00:32:50.054250 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:34589
I20260808 00:32:50.060012 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.210.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.253.keytab.
Entry for principal oryx/127.31.210.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.253.keytab.
I20260808 00:32:50.087745 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:45967
--webserver_interface=127.31.210.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.253.keytab
--principal=oryx/127.31.210.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:34983,127.31.210.253:45967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:50.180428 9655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:50.180773 9655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:50.180811 9655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:50.181964 9655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:50.182040 9655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:50.182066 9655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:50.182087 9655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:50.182121 9655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:50.183583 9655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:34983,127.31.210.253:45967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.253.keytab
--principal=oryx/127.31.210.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:45967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.210.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
W20260808 00:32:50.183736 9655 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.210.254:34983,127.31.210.253:45967'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 00:32:50.184033 9655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:50.184278 9655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:50.189334 9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.189343 9660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.189343 9661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.189626 9655 server_base.cc:1034] running on GCE node
I20260808 00:32:50.189894 9655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:50.190173 9655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:50.191346 9655 hybrid_clock.cc:648] HybridClock initialized: now 1786149170191314 us; error 43 us; skew 500 ppm
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:50.194551 9655 init.cc:377] Logged in from keytab as oryx/127.31.210.253@KRBTEST.COM (short username oryx)
I20260808 00:32:50.195055 9655 webserver.cc:460] Webserver started at http://127.31.210.253:35325/ using document root <none> and password file <none>
I20260808 00:32:50.195282 9655 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.195328 9655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.195498 9655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:50.196666 9655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/instance:
uuid: "e727e1926bcd41e6bc94090828fe4263"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "43c854c6182cf19bec1e45f75c88f851"
server_key_iv: "8e6a4b63289555f4822e3b3c593737ef"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.197036 9655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal/instance:
uuid: "e727e1926bcd41e6bc94090828fe4263"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "43c854c6182cf19bec1e45f75c88f851"
server_key_iv: "8e6a4b63289555f4822e3b3c593737ef"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.198482 9655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:32:50.199277 9673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.199450 9655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:50.199549 9655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
uuid: "e727e1926bcd41e6bc94090828fe4263"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "43c854c6182cf19bec1e45f75c88f851"
server_key_iv: "8e6a4b63289555f4822e3b3c593737ef"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.199635 9655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.207206 9655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:50.209405 9655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:50.209523 9655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:50.214244 9655 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.253:45967
I20260808 00:32:50.214345 9739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.253:45967 every 8 connection(s)
I20260808 00:32:50.214846 9655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
I20260808 00:32:50.217625 9740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:50.219297 9740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:32:50.223822 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 9655
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.253@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.210.253@KRBTEST.COM
I20260808 00:32:50.230918 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.220099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.31.210.254:34983)
0808 00:32:50.220175 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.220178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.220406 (+ 228us) server_negotiation.cc:407] Connection header received
0808 00:32:50.221049 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.221050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.221063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.221124 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.221844 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.222877 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.224085 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.224336 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.224437 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.227246 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.227248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.227249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.227258 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.229120 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.229621 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.229622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.229623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.229694 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.230029 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.230030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.230031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.230287 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.230371 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.230692 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.230790 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:32:50.231050 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.220065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:59818)
0808 00:32:50.220258 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.220307 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.220955 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.221180 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.221183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.221462 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.221675 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.221679 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.223018 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.223020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.223957 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.223959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.224562 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.224564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.224723 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.225427 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.225437 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.227136 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.229237 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.229246 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.229248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.229529 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.229817 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.229818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.229826 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.229947 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.230366 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.230372 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.230522 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.230633 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.230762 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260808 00:32:50.231861 9740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:32:50.235069 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.224503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.31.210.253:45967)
0808 00:32:50.224883 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.224889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.224920 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:32:50.224945 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.224947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.224970 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.225059 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.225810 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.226693 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.227547 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.227892 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.227989 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.231417 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.231424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.231424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.231432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.233650 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.234066 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.234069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.234069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.234137 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.234373 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.234374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.234374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.234551 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.234749 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.234806 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.234925 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":289,"threads_started":1}
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.253@KRBTEST.COM for oryx/127.31.210.253@KRBTEST.COM
I20260808 00:32:50.239847 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.232181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.0.0.1:33518)
0808 00:32:50.232240 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.232255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.232291 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.232948 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.232949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.233108 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.233179 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.233181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.233824 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.233825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.234323 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.234325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.234715 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.234716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.234779 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.235350 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.235354 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.236655 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.238796 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.238797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.238797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.239002 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.239210 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.239211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.239211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.239274 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.239592 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.239593 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.239633 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.239702 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.239785 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:32:50.239938 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.232224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33518 (local address 127.31.210.253:45967)
0808 00:32:50.232812 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.232814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.232821 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:32:50.232838 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.232839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.232847 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.232908 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.233270 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.233736 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.234422 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.234641 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.234718 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.236736 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.236737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.236738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.236743 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.238710 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.239073 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.239074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.239074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.239128 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.239343 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.239344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.239344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.239440 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.239497 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.239734 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.239839 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":568,"thread_start_us":241,"threads_started":1}
I20260808 00:32:50.241468 9740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Bootstrap starting.
I20260808 00:32:50.242264 9740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:50.242592 9740 log.cc:824] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:50.243366 9740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: No bootstrap required, opened a new log
I20260808 00:32:50.244937 9740 raft_consensus.cc:348] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } }
I20260808 00:32:50.245061 9740 raft_consensus.cc:374] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:50.245085 9740 raft_consensus.cc:729] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e727e1926bcd41e6bc94090828fe4263, State: Initialized, Role: FOLLOWER
I20260808 00:32:50.245213 9740 consensus_queue.cc:260] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } }
I20260808 00:32:50.245528 9740 sys_catalog.cc:564] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:50.245504 9749 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } } }
I20260808 00:32:50.245617 9749 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:32:50.247236 9766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:32:50.247298 9766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:32:50.249384 9655 master_runner.cc:428] Detected that this master e727e1926bcd41e6bc94090828fe4263 is joining an existing cluster
I20260808 00:32:50.249533 9655 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.210.253:45967
I20260808 00:32:50.259145 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.250742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:59828)
0808 00:32:50.250950 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.250964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.251194 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.251337 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.251338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.251612 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.251684 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.251687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.252823 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.252825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.253885 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.253888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.254361 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.254363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.254440 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.254967 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.254973 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.255410 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.257735 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.257736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.257737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.258066 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.258314 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.258314 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.258315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.258393 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.258712 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.258713 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.258759 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.258858 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.258981 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":47,"thread_start_us":124,"threads_started":1}
I20260808 00:32:50.259136 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.250787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.31.210.254:34983)
0808 00:32:50.251075 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.251077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.251214 (+ 137us) server_negotiation.cc:407] Connection header received
0808 00:32:50.251257 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.251258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.251270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.251332 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.251779 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.252713 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.254012 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.254231 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.254318 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.255530 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.255532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.255532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.255541 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.257631 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.258163 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.258164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.258165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.258236 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.258471 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.258472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.258473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.258645 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.258724 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.258888 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.258981 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258}
I20260808 00:32:50.260237 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.250770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.0.0.1:33534)
0808 00:32:50.251187 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.251197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.251349 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.251513 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.251514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.251724 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.251793 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.251795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.252839 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.252840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.253886 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.253888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.254457 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.254458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.254516 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.255171 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.255176 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.255583 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.258026 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.258027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.258027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.258290 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.258496 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.258497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.258497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.258568 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.259572 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.259573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.259857 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.259949 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.260103 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":37,"thread_start_us":212,"threads_started":1}
I20260808 00:32:50.260316 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.251120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33534 (local address 127.31.210.253:45967)
0808 00:32:50.251189 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.251191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.251324 (+ 133us) server_negotiation.cc:407] Connection header received
0808 00:32:50.251422 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.251423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.251434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.251494 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.251862 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.252756 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.253981 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.254323 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.254404 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.255672 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.255673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.255673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.255681 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.257926 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.258352 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.258353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.258353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.258417 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.258640 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.258641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.258642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.258774 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.258847 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.259976 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.260184 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":45}
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.254@KRBTEST.COM for oryx/127.31.210.253@KRBTEST.COM
I20260808 00:32:50.269294 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.261932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33536 (local address 127.31.210.253:45967)
0808 00:32:50.261979 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.261982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.262141 (+ 159us) server_negotiation.cc:407] Connection header received
0808 00:32:50.262237 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.262238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.262250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.262299 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.262753 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.263257 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.264056 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.264262 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.264340 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.266604 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.266606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.266606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.266613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.267941 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.268384 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.268385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.268386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.268449 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.268725 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.268726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.268727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.268844 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.268922 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.269110 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.269193 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:32:50.269317 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.261836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.0.0.1:33536)
0808 00:32:50.262022 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.262075 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.262167 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.262302 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.262306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.262609 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.262665 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.262667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.263344 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.263346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.263966 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.263968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.264339 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.264340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.264426 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.264971 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.264976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.266501 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.268083 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.268092 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.268094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.268305 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.268564 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.268565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.268566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.268655 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.268914 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.268922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.268980 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.269083 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.269207 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260808 00:32:50.270190 9008 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.210.253:45967: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e727e1926bcd41e6bc94090828fe4263"
member_type: NON_VOTER
last_known_addr {
host: "127.31.210.253"
port: 45967
}
attrs {
promote: true
}
}
dest_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6"
cas_config_opid_index: -1
I20260808 00:32:50.270586 9008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } }
I20260808 00:32:50.271091 9782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e727e1926bcd41e6bc94090828fe4263 (127.31.210.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } }
I20260808 00:32:50.271505 9705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:50.272271 9782 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.210.253:45967
I20260808 00:32:50.272369 9781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } }
I20260808 00:32:50.272464 9781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:50.272549 9705 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Refusing update from remote peer 8c3ba4458c064fc6bc5369f6b30a6ef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 00:32:50.272969 9655 master.cc:537] Master@127.31.210.253:45967 shutting down...
I20260808 00:32:50.272936 9781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:32:50.273686 8986 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 00:32:50.274240 8986 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 -> Peer e727e1926bcd41e6bc94090828fe4263 (127.31.210.253:45967): Couldn't send request to peer e727e1926bcd41e6bc94090828fe4263. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:32:50.276262 9655 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:50.276357 9655 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:50.276424 9655 tablet_replica.cc:330] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: stopping tablet replica
I20260808 00:32:50.289911 9655 master.cc:552] Master@127.31.210.253:45967 shutdown complete.
I20260808 00:32:50.289999 9655 master_runner.cc:305] Clearing existing system tablet
I20260808 00:32:50.290138 9655 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.290202 9655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.290871 9655 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260808 00:32:50.291389 9789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.291519 9655 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.291579 9655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.291623 9655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
uuid: "e727e1926bcd41e6bc94090828fe4263"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "43c854c6182cf19bec1e45f75c88f851"
server_key_iv: "8e6a4b63289555f4822e3b3c593737ef"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.292426 9655 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:50.294116 9655 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 00:32:50.294178 9655 log.cc:1196] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 00:32:50.294381 9655 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Deleting consensus metadata
I20260808 00:32:50.294523 9655 master_runner.cc:315] Copying system tablet from 127.31.210.254:34983
I20260808 00:32:50.295224 9655 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: tablet copy: Beginning tablet copy session from remote peer at address 127.31.210.254:34983
I20260808 00:32:50.302160 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.295704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.31.210.254:34983)
0808 00:32:50.295755 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.295761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.295819 (+ 58us) server_negotiation.cc:407] Connection header received
0808 00:32:50.295903 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.295904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.295913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.295965 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.296370 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.296965 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.298039 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.298222 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.298268 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.299440 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.299441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.299441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.299447 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.301049 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.301444 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.301444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.301445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.301513 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.301711 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.301712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.301712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.301818 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.301867 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.302031 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.302095 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:32:50.302230 9795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.295598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:59844)
0808 00:32:50.295750 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.295762 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.295837 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.295986 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.295987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.296208 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.296285 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.296287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.297046 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.297047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.297950 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.297953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.298291 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.298293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.298366 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.298929 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.298934 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.299345 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.301130 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.301131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.301132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.301363 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.301576 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.301577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.301577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.301647 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.301880 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.301880 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.301930 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.302003 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.302114 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260808 00:32:50.302630 9035 tablet_copy_service.cc:140] P 8c3ba4458c064fc6bc5369f6b30a6ef6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e727e1926bcd41e6bc94090828fe4263 ({username='oryx', principal='oryx/127.31.210.253@KRBTEST.COM'} at 127.0.0.1:59844)
I20260808 00:32:50.302784 9035 tablet_copy_service.cc:161] P 8c3ba4458c064fc6bc5369f6b30a6ef6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e727e1926bcd41e6bc94090828fe4263 at {username='oryx', principal='oryx/127.31.210.253@KRBTEST.COM'} at 127.0.0.1:59844: session id = e727e1926bcd41e6bc94090828fe4263-00000000000000000000000000000000
I20260808 00:32:50.303771 9035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 00:32:50.304399 9655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 00:32:50.305451 9655 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: tablet copy: Starting download of 0 data blocks...
I20260808 00:32:50.305629 9655 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: tablet copy: Starting download of 1 WAL segments...
I20260808 00:32:50.306739 9655 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 00:32:50.307729 9035 tablet_copy_service.cc:342] P 8c3ba4458c064fc6bc5369f6b30a6ef6: Request end of tablet copy session e727e1926bcd41e6bc94090828fe4263-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.210.253@KRBTEST.COM'} at 127.0.0.1:59844
I20260808 00:32:50.307796 9035 tablet_copy_service.cc:433] P 8c3ba4458c064fc6bc5369f6b30a6ef6: ending tablet copy session e727e1926bcd41e6bc94090828fe4263-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e727e1926bcd41e6bc94090828fe4263
I20260808 00:32:50.310592 9655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:50.311887 9798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.312224 9796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.312685 9655 server_base.cc:1034] running on GCE node
W20260808 00:32:50.313501 9800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.313669 9655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:50.313920 9655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:50.315088 9655 hybrid_clock.cc:648] HybridClock initialized: now 1786149170315063 us; error 41 us; skew 500 ppm
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:50.316756 9655 init.cc:377] Logged in from keytab as oryx/127.31.210.253@KRBTEST.COM (short username oryx)
I20260808 00:32:50.317090 9655 webserver.cc:460] Webserver started at http://127.31.210.253:41651/ using document root <none> and password file <none>
I20260808 00:32:50.317222 9655 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.317282 9655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.317972 9655 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:50.318578 9808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.318724 9655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.318775 9655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
uuid: "e727e1926bcd41e6bc94090828fe4263"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "43c854c6182cf19bec1e45f75c88f851"
server_key_iv: "8e6a4b63289555f4822e3b3c593737ef"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.318838 9655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.330952 9655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:50.332703 9655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:50.335959 9655 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.253:45967
I20260808 00:32:50.335990 9880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.253:45967 every 8 connection(s)
I20260808 00:32:50.336300 9655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
I20260808 00:32:50.337337 9881 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:32:50.337904 9881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Bootstrap starting.
I20260808 00:32:50.341564 9881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:32:50.341888 9881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: Bootstrap complete.
I20260808 00:32:50.342098 9881 raft_consensus.cc:348] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } }
I20260808 00:32:50.342190 9881 raft_consensus.cc:729] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e727e1926bcd41e6bc94090828fe4263, State: Initialized, Role: LEARNER
I20260808 00:32:50.342235 9881 consensus_queue.cc:260] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } }
I20260808 00:32:50.342425 9888 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } }
I20260808 00:32:50.342558 9888 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:32:50.342469 9881 sys_catalog.cc:564] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:50.344419 9900 catalog_manager.cc:1259] Loaded cluster ID: 74dd0d7715f042418d187f293141dee1
I20260808 00:32:50.344632 9900 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: loading cluster ID for follower catalog manager: success
I20260808 00:32:50.344762 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.336677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.31.210.253:45967)
0808 00:32:50.336835 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.336838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.336846 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:50.336874 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.336876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.336888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.337019 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.337629 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.338347 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.339073 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.339285 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.339365 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.340932 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.340934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.340935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.340943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.342944 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.343362 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.343363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.343364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.343426 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.343653 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.343654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.343655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.343791 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.343884 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.344545 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.344649 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260808 00:32:50.345849 9900 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: acquiring CA information for follower catalog manager: success
I20260808 00:32:50.346195 9900 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for oryx/127.31.210.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.252.keytab.
Entry for principal oryx/127.31.210.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.252.keytab.
I20260808 00:32:50.376433 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:38835
--webserver_interface=127.31.210.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.252.keytab
--principal=oryx/127.31.210.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:50.464293 9905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:50.464538 9905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:50.464563 9905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:50.465734 9905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:50.465780 9905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:50.465793 9905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:50.465806 9905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:50.465818 9905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:50.467360 9905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.252.keytab
--principal=oryx/127.31.210.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:38835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.210.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:50.467679 9905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:50.467932 9905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:50.472994 9913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.473060 9916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.473086 9911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.473347 9905 server_base.cc:1034] running on GCE node
I20260808 00:32:50.473592 9905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:50.473845 9905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:50.474992 9905 hybrid_clock.cc:648] HybridClock initialized: now 1786149170474976 us; error 32 us; skew 500 ppm
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:50.477757 9905 init.cc:377] Logged in from keytab as oryx/127.31.210.252@KRBTEST.COM (short username oryx)
I20260808 00:32:50.478307 9905 webserver.cc:460] Webserver started at http://127.31.210.252:46427/ using document root <none> and password file <none>
I20260808 00:32:50.478560 9905 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.478633 9905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.478786 9905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:50.479882 9905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/instance:
uuid: "54d2c0b4907e42e1aec13b2b994bffa3"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "5d344cf97a844fb3e2032d84cd6f36b6"
server_key_iv: "0e370ae9129d7e0b3bfe92ef35b1c657"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.480257 9905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal/instance:
uuid: "54d2c0b4907e42e1aec13b2b994bffa3"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "5d344cf97a844fb3e2032d84cd6f36b6"
server_key_iv: "0e370ae9129d7e0b3bfe92ef35b1c657"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.481722 9905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:50.482654 9926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.482963 9905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:50.483076 9905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
uuid: "54d2c0b4907e42e1aec13b2b994bffa3"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "5d344cf97a844fb3e2032d84cd6f36b6"
server_key_iv: "0e370ae9129d7e0b3bfe92ef35b1c657"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.483152 9905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.492676 9905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:50.495271 9905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:50.495447 9905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:50.500607 9905 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.252:38835
I20260808 00:32:50.500643 10000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.252:38835 every 8 connection(s)
I20260808 00:32:50.501029 9905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
I20260808 00:32:50.502051 10001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:50.503656 10001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:32:50.503806 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 9905
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.252@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.210.252@KRBTEST.COM
I20260808 00:32:50.514474 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.504648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.31.210.254:34983)
0808 00:32:50.504703 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.504705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.504858 (+ 153us) server_negotiation.cc:407] Connection header received
0808 00:32:50.505476 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.505478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.505492 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.505564 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.506229 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.506738 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.507752 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.508015 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.508069 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.510696 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.510698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.510698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.510726 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.512718 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.513255 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.513257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.513257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.513336 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.513626 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.513627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.513628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.513770 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.513874 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.514252 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.514362 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:32:50.514798 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.504387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:59850)
0808 00:32:50.504727 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.504772 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.505347 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.505598 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.505601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.505867 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.506129 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.506136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.506836 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.506837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.507671 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.507673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.508109 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.508111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.508223 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.508999 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.509008 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.510580 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.512814 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.512820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.512822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.513165 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.513424 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.513425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.513425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.513524 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.513850 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.513857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.514065 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.514199 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.514338 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":254,"threads_started":1}
I20260808 00:32:50.514971 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.504767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.31.210.252:38835)
0808 00:32:50.505011 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.505016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.505055 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:32:50.505350 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.505359 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.505388 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.505477 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.506157 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.507166 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.507917 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.508313 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.508389 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.511177 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.511186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.511186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.511193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.513511 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.513896 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.513900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.513900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.513973 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.514221 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.514222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.514223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.514457 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.514688 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.514733 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.514804 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":150,"threads_started":1}
I20260808 00:32:50.515594 10001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.252@KRBTEST.COM for oryx/127.31.210.253@KRBTEST.COM
I20260808 00:32:50.522740 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.515912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.0.0.1:33582)
0808 00:32:50.515963 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.515973 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.516014 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.516239 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.516240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.516355 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.516387 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.516388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.517268 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.517269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.517801 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.517802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.518319 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.518320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.518370 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.518864 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.518868 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.520220 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.521942 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.521943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.521943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.522105 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.522288 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.522288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.522289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.522338 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.522517 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.522518 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.522553 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.522603 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.522676 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:32:50.522750 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.516032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.31.210.253:45967)
0808 00:32:50.516099 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.516101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.516117 (+ 16us) server_negotiation.cc:407] Connection header received
0808 00:32:50.516156 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.516157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.516171 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.516236 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.516530 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.517171 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.517940 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.518220 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.518301 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.520313 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.520315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.520315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.520322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.521860 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.522172 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.522172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.522173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.522228 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.522375 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.522375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.522375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.522468 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.522513 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.522620 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.522688 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:32:50.523284 10001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.252@KRBTEST.COM for oryx/127.31.210.252@KRBTEST.COM
I20260808 00:32:50.529592 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.523572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.0.0.1:56000)
0808 00:32:50.523622 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.523632 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.523671 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.523780 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.523781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.523888 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.523924 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.523925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.524576 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.524577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.525094 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.525095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.525425 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.525426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.525462 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.525781 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.525785 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.527081 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.528779 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.528780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.528781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.528951 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.529131 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.529132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.529132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.529182 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.529352 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.529353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.529384 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.529440 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.529514 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:32:50.529649 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.523643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.31.210.252:38835)
0808 00:32:50.523680 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.523682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.523714 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:32:50.523730 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.523731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.523741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.523778 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.524006 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.524451 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.525165 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.525361 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.525412 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.527169 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.527170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.527170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.527176 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.528709 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.529022 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.529023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.529023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.529074 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.529220 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.529220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.529220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.529304 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.529348 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.529460 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.529559 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":20}
I20260808 00:32:50.531173 10001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Bootstrap starting.
I20260808 00:32:50.532316 10001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:50.532718 10001 log.cc:824] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:50.533699 10001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: No bootstrap required, opened a new log
I20260808 00:32:50.535526 10001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } }
I20260808 00:32:50.535651 10001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:50.535673 10001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d2c0b4907e42e1aec13b2b994bffa3, State: Initialized, Role: FOLLOWER
I20260808 00:32:50.535799 10001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } }
I20260808 00:32:50.536072 10007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } } }
I20260808 00:32:50.536132 10001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:50.536180 10007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:32:50.537830 10021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:32:50.537901 10021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:32:50.540570 9905 master_runner.cc:428] Detected that this master 54d2c0b4907e42e1aec13b2b994bffa3 is joining an existing cluster
I20260808 00:32:50.540649 9905 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.210.252:38835
I20260808 00:32:50.550248 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.542403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.0.0.1:56004)
0808 00:32:50.542629 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.542640 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.542752 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.542925 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.542927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.543293 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.543362 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.543363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.544371 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.544371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.545388 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.545389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.545875 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.545941 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.546914 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.546919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.547282 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.549116 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.549117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.549117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.549302 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.549554 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.549555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.549556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.549635 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.549973 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.549974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.550013 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.550071 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.550166 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":101,"thread_start_us":83,"threads_started":1}
I20260808 00:32:50.550348 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.542357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.31.210.252:38835)
0808 00:32:50.542392 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.542393 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.542700 (+ 307us) server_negotiation.cc:407] Connection header received
0808 00:32:50.542822 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.542823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.542833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.542863 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.543449 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.544230 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545470 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.545742 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545781 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.547388 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.547390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.547390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.547398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.549019 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.549383 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.549384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.549385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.549459 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.549712 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.549713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.549714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.549854 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.549965 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.550121 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.550235 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":19}
I20260808 00:32:50.567039 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 9096
I20260808 00:32:50.571655 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.542216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.0.0.1:33584)
0808 00:32:50.542604 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.542615 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.542708 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.542985 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.542986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.543336 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.543425 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.543427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.544368 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.544369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.545466 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.545470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.546052 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.546053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.546120 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.565236 (+ 19116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.565241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.565829 (+ 588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.570324 (+ 4495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.570325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.570326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.570562 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.570833 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.570834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.570834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.570903 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.571264 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.571265 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.571329 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.571405 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.571533 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":48,"thread_start_us":202,"threads_started":1}
I20260808 00:32:50.571743 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.542572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33584 (local address 127.31.210.253:45967)
0808 00:32:50.542620 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.542622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.542678 (+ 56us) server_negotiation.cc:407] Connection header received
0808 00:32:50.542823 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.542824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.542836 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.542867 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.543528 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.544267 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545576 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.545916 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545964 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.565997 (+ 20033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.565999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.566000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.566010 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.570164 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.570655 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.570657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.570658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.570745 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.570981 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.570982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.570983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.571181 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.571272 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.571469 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.571588 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:32:50.580521 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.542451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:59862)
0808 00:32:50.542671 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.542680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.542751 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.542931 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.542932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.543222 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.543359 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.543361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.544307 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.544308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.545368 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.545370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.545904 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.545960 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.565171 (+ 19211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.565177 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.565758 (+ 581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.579154 (+ 13396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.579155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.579156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.579408 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.579690 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.579691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.579691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.579765 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.580089 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.580090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.580143 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.580220 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.580343 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":89,"thread_start_us":83,"threads_started":1}
I20260808 00:32:50.580576 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.542276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.31.210.254:34983)
0808 00:32:50.542321 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.542323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.542727 (+ 404us) server_negotiation.cc:407] Connection header received
0808 00:32:50.542828 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.542829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.542842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.542871 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.543469 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.544205 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545487 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.545789 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.545829 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.565909 (+ 20080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.565915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.565916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.565924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.578985 (+ 13061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.579507 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.579509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.579510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.579595 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.579841 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.579842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.579842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.580012 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.580089 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.580271 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.580379 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:32:50.585498 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:35275
--local_ip_for_outbound_sockets=127.31.210.193
--tserver_master_addrs=127.31.210.254:34983
--webserver_port=40747
--webserver_interface=127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab
--principal=oryx/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149169, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.254@KRBTEST.COM for oryx/127.31.210.252@KRBTEST.COM
I20260808 00:32:50.591490 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.582996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.0.0.1:56014)
0808 00:32:50.583094 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.583104 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.583149 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.583327 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.583329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.583528 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.583584 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.583586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.584544 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.584545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.585292 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.585296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.585812 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.585813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.585894 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.586392 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.586397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.588053 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.590191 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.590192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.590193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.590441 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.590687 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.590688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.590689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.590757 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.591074 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.591076 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.591127 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.591194 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.591319 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 00:32:50.592231 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.583106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56014 (local address 127.31.210.252:38835)
0808 00:32:50.583150 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.583152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.583162 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:50.583237 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.583238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.583249 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.583311 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.583669 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.584398 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.585395 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.585670 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.585746 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.588156 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.588158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.588158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.588166 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.590093 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.590547 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.590549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.590550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.590613 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.590828 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.590829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.590829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.590972 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.591041 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.592006 (+ 965us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.592098 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:32:50.592831 9008 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.210.252:38835: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3"
member_type: NON_VOTER
last_known_addr {
host: "127.31.210.252"
port: 38835
}
attrs {
promote: true
}
}
dest_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6"
cas_config_opid_index: 5
I20260808 00:32:50.593099 9008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } }
I20260808 00:32:50.593508 9782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.593911 9965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:50.594494 9782 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.210.252:38835
I20260808 00:32:50.594556 9782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.594617 9782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:50.595063 9905 master.cc:537] Master@127.31.210.252:38835 shutting down...
I20260808 00:32:50.595268 9965 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Refusing update from remote peer 8c3ba4458c064fc6bc5369f6b30a6ef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 00:32:50.595700 9781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:32:50.596553 8988 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 -> Peer 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252:38835): Couldn't send request to peer 54d2c0b4907e42e1aec13b2b994bffa3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:32:50.598181 9905 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:50.598287 9905 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:50.598335 9905 tablet_replica.cc:330] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: stopping tablet replica
I20260808 00:32:50.600556 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.594808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.0.0.1:33600)
0808 00:32:50.595430 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.595442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.595493 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.595728 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.595729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.595731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:50.595774 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.595775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.597889 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.597890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.599216 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.599219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.600266 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.600268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.600337 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.600362 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.600424 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.600467 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594}
I20260808 00:32:50.600809 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.594948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.31.210.253:45967)
0808 00:32:50.595383 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.595386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.595548 (+ 162us) server_negotiation.cc:407] Connection header received
0808 00:32:50.595577 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.595578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.595607 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.595666 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:50.595999 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.597785 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.599334 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.600077 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.600150 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.600218 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.600703 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.600744 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404}
I20260808 00:32:50.601044 9847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 LEARNER]: Refusing update from remote peer 8c3ba4458c064fc6bc5369f6b30a6ef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 00:32:50.601284 10033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 00:32:50.601929 9847 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.603302 9888 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3ba4458c064fc6bc5369f6b30a6ef6. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.603356 9781 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: attempting to promote NON_VOTER e727e1926bcd41e6bc94090828fe4263 to VOTER
I20260808 00:32:50.603390 9888 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:32:50.603428 9888 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: NON_VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: true } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.603499 9888 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:32:50.603564 9781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } }
I20260808 00:32:50.604067 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.603934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.0.0.1:56018)
0808 00:32:50.603987 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.604038 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:38835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:32:50.604355 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.604287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.0.0.1:56024)
0808 00:32:50.604316 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.604336 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:38835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 00:32:50.604405 8988 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 -> Peer 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252:38835): Couldn't send request to peer 54d2c0b4907e42e1aec13b2b994bffa3. Status: Network error: Client connection negotiation failed: client connection to 127.31.210.252:38835: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 00:32:50.605139 9847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 LEARNER]: Refusing update from remote peer 8c3ba4458c064fc6bc5369f6b30a6ef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 00:32:50.605360 10037 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 00:32:50.606097 10033 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e727e1926bcd41e6bc94090828fe4263 (127.31.210.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.606266 9847 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e727e1926bcd41e6bc94090828fe4263 (127.31.210.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.607196 10035 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.607335 10035 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:32:50.607530 10037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.607553 9782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.607664 9782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:50.607733 10037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:50.612282 9905 master.cc:552] Master@127.31.210.252:38835 shutdown complete.
I20260808 00:32:50.612336 9905 master_runner.cc:305] Clearing existing system tablet
I20260808 00:32:50.612498 9905 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.612557 9905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.613344 9905 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:50.613960 10040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.614118 9905 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.614173 9905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:50.614197 9905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
uuid: "54d2c0b4907e42e1aec13b2b994bffa3"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "5d344cf97a844fb3e2032d84cd6f36b6"
server_key_iv: "0e370ae9129d7e0b3bfe92ef35b1c657"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.614934 9905 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:50.616550 9905 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 00:32:50.616654 9905 log.cc:1196] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 00:32:50.616883 9905 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Deleting consensus metadata
I20260808 00:32:50.617034 9905 master_runner.cc:315] Copying system tablet from 127.31.210.254:34983
I20260808 00:32:50.617784 9905 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: tablet copy: Beginning tablet copy session from remote peer at address 127.31.210.254:34983
I20260808 00:32:50.625516 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.618203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.31.210.254:34983)
0808 00:32:50.618252 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.618254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.618357 (+ 103us) server_negotiation.cc:407] Connection header received
0808 00:32:50.618453 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.618454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.618473 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.618548 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.618957 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.619756 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.620839 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.621134 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.621211 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.622453 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.622455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.622455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.622469 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.624158 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.624638 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.624639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.624640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.624705 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.624930 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.624931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.624931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.625066 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.625138 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.625330 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.625413 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:32:50.625525 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.618135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:59874)
0808 00:32:50.618292 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.618304 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.618374 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.618568 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.618569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.618799 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.618860 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.618862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.619855 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.619856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.620754 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.620757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.621224 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.621225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.621306 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.621928 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.621933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.622358 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.624255 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.624256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.624257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.624563 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.624785 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.624786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.624786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.624862 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.625149 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.625150 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.625213 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.625296 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.625423 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260808 00:32:50.626093 9035 tablet_copy_service.cc:140] P 8c3ba4458c064fc6bc5369f6b30a6ef6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 54d2c0b4907e42e1aec13b2b994bffa3 ({username='oryx', principal='oryx/127.31.210.252@KRBTEST.COM'} at 127.0.0.1:59874)
I20260808 00:32:50.626150 9035 tablet_copy_service.cc:161] P 8c3ba4458c064fc6bc5369f6b30a6ef6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 54d2c0b4907e42e1aec13b2b994bffa3 at {username='oryx', principal='oryx/127.31.210.252@KRBTEST.COM'} at 127.0.0.1:59874: session id = 54d2c0b4907e42e1aec13b2b994bffa3-00000000000000000000000000000000
I20260808 00:32:50.626848 9035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 00:32:50.627383 9905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 00:32:50.628865 9905 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: tablet copy: Starting download of 0 data blocks...
I20260808 00:32:50.629034 9905 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: tablet copy: Starting download of 1 WAL segments...
I20260808 00:32:50.629838 9905 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 00:32:50.630764 9035 tablet_copy_service.cc:342] P 8c3ba4458c064fc6bc5369f6b30a6ef6: Request end of tablet copy session 54d2c0b4907e42e1aec13b2b994bffa3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.210.252@KRBTEST.COM'} at 127.0.0.1:59874
I20260808 00:32:50.630813 9035 tablet_copy_service.cc:433] P 8c3ba4458c064fc6bc5369f6b30a6ef6: ending tablet copy session 54d2c0b4907e42e1aec13b2b994bffa3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 54d2c0b4907e42e1aec13b2b994bffa3
I20260808 00:32:50.633500 9905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:50.634814 10048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.637482 10051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.637588 9905 server_base.cc:1034] running on GCE node
W20260808 00:32:50.640801 10049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.641033 9905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:50.641177 9905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:50.642297 9905 hybrid_clock.cc:648] HybridClock initialized: now 1786149170642269 us; error 44 us; skew 500 ppm
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:50.643887 9905 init.cc:377] Logged in from keytab as oryx/127.31.210.252@KRBTEST.COM (short username oryx)
I20260808 00:32:50.644156 9905 webserver.cc:460] Webserver started at http://127.31.210.252:34887/ using document root <none> and password file <none>
I20260808 00:32:50.644274 9905 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.644308 9905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.645143 9905 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:50.649885 10062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.650007 9905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:50.650038 9905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
uuid: "54d2c0b4907e42e1aec13b2b994bffa3"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "5d344cf97a844fb3e2032d84cd6f36b6"
server_key_iv: "0e370ae9129d7e0b3bfe92ef35b1c657"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.650097 9905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.665688 9905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:50.667614 9905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:50.671187 9905 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.252:38835
I20260808 00:32:50.671231 10148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.252:38835 every 8 connection(s)
I20260808 00:32:50.671547 9905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
I20260808 00:32:50.672670 10149 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:32:50.673164 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Bootstrap starting.
I20260808 00:32:50.676164 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:32:50.676391 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Bootstrap complete.
I20260808 00:32:50.676684 10149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } }
I20260808 00:32:50.676819 10149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54d2c0b4907e42e1aec13b2b994bffa3, State: Initialized, Role: LEARNER
I20260808 00:32:50.676864 10149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } }
I20260808 00:32:50.677073 10149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:50.677060 10154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:50.677299 10154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:32:50.678747 10170 catalog_manager.cc:1259] Loaded cluster ID: 74dd0d7715f042418d187f293141dee1
I20260808 00:32:50.678838 10170 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: loading cluster ID for follower catalog manager: success
I20260808 00:32:50.680011 10170 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: acquiring CA information for follower catalog manager: success
I20260808 00:32:50.680305 10170 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 00:32:50.700102 10032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:50.700340 10032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:50.700379 10032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:50.701622 10032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:50.701691 10032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:50.701740 10032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:50.703346 10032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab
--principal=oryx/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:35275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=40747
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:50.703625 10032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:50.703874 10032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:50.709241 10180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.709401 10177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.709429 10178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.709911 10032 server_base.cc:1034] running on GCE node
I20260808 00:32:50.710177 10032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:50.710431 10032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:50.711578 10032 hybrid_clock.cc:648] HybridClock initialized: now 1786149170711562 us; error 26 us; skew 500 ppm
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:50.714444 10032 init.cc:377] Logged in from keytab as oryx/127.31.210.193@KRBTEST.COM (short username oryx)
I20260808 00:32:50.714942 10032 webserver.cc:460] Webserver started at http://127.31.210.193:40747/ using document root <none> and password file <none>
I20260808 00:32:50.715224 10032 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.715327 10032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.717042 10032 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:32:50.717736 10190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.717911 10032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:50.718086 10032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "a690268c10b748f7995fed5d1bd9a483"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "236c1bebed27a98846484d8958e1b8d6"
server_key_iv: "5ff23e09df0cbd631ae032f61a5e3975"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.718444 10032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.729161 10032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:50.731079 10032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:50.731254 10032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:50.731498 10032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:50.731825 10032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:50.731884 10032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.731925 10032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:50.731951 10032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.741943 10032 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:35275
I20260808 00:32:50.742056 10341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:35275 every 8 connection(s)
I20260808 00:32:50.742481 10032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:50.745774 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 10032
I20260808 00:32:50.745968 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 9288
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.252@KRBTEST.COM
I20260808 00:32:50.757995 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.743762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:44103 (local address 127.31.210.252:38835)
0808 00:32:50.743910 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.743913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.743925 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:50.745117 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.745118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.745133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.745220 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.748601 (+ 3381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.749849 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.751333 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.751632 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.754380 (+ 2748us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.754433 (+ 53us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.754433 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.754434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.754438 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.756186 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.756717 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.756718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.756719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.756838 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.757155 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.757156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.757156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.757294 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.757735 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.757790 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.757879 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260808 00:32:50.758122 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.743366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.31.210.193:44103)
0808 00:32:50.743563 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.743597 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.744939 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.745249 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.745252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.745840 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.748407 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.748418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.750058 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.750059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.751231 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.751234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.751715 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.751716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.751867 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.752472 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.752500 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.754286 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.756281 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.756290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.756293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.756632 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.756917 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.756918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.756918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.757076 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.757362 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.757369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.757541 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.757634 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.757875 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":66,"thread_start_us":130,"threads_started":1}
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.253@KRBTEST.COM
I20260808 00:32:50.763870 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.743993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:36239 (local address 127.31.210.253:45967)
0808 00:32:50.744053 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.744055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.744064 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:50.745084 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.745086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.745100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.745155 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.756730 (+ 11575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.757486 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.758837 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.759060 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.759155 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.761136 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.761138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.761138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.761146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.762724 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.763096 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.763098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.763098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.763148 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.763307 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.763307 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.763308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.763443 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.763492 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.763655 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.763736 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:32:50.764221 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.743707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.31.210.193:36239)
0808 00:32:50.743896 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.743906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.744948 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.745247 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.745252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.745895 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.756596 (+ 10701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.756601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.757583 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.757584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.758739 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.758742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.759157 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.759158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.759233 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.759703 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.759708 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.761044 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.762816 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.762817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.762817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.763027 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.763204 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.763204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.763205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.763254 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.763521 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.763522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.763562 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.764004 (+ 442us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.764117 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":7,"thread_start_us":151,"threads_started":1}
I20260808 00:32:50.764621 10342 heartbeater.cc:347] Connected to a master server at 127.31.210.252:38835
I20260808 00:32:50.764727 10342 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:50.764894 10342 heartbeater.cc:511] Master 127.31.210.252:38835 requested a full tablet report, sending...
I20260808 00:32:50.765430 10089 ts_manager.cc:194] Registered new tserver with Master: a690268c10b748f7995fed5d1bd9a483 (127.31.210.193:35275)
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:50.771186 10344 heartbeater.cc:347] Connected to a master server at 127.31.210.253:45967
I20260808 00:32:50.771260 10344 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:50.771351 10344 heartbeater.cc:511] Master 127.31.210.253:45967 requested a full tablet report, sending...
I20260808 00:32:50.773572 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.743343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:54745 (local address 127.31.210.254:34983)
0808 00:32:50.743402 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.743404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.743686 (+ 282us) server_negotiation.cc:407] Connection header received
0808 00:32:50.745076 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.745077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.745092 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.745124 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.766436 (+ 21312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.766920 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.768424 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.768685 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.768768 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.770757 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.770759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.770759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.770768 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.772320 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.772743 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.772745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.772745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.772809 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.773029 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.773030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.773030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.773144 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.773223 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.773385 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.773470 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:32:50.773581 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.743366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:54745)
0808 00:32:50.743538 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.743617 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.744940 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.745247 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.745252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.766234 (+ 20982us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.766299 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.766303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.767018 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.767019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.768307 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.768311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.768768 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.768769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.768836 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.769319 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.769324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.770624 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.772427 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.772428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.772429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.772653 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.772885 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.772886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.772887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.772952 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.773215 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.773216 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.773259 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.773342 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.773450 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
I20260808 00:32:50.773947 10343 heartbeater.cc:347] Connected to a master server at 127.31.210.254:34983
I20260808 00:32:50.774003 10343 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:50.774056 10343 heartbeater.cc:511] Master 127.31.210.254:34983 requested a full tablet report, sending...
I20260808 00:32:50.774324 9008 ts_manager.cc:194] Re-registered known tserver with Master: a690268c10b748f7995fed5d1bd9a483 (127.31.210.193:35275)
I20260808 00:32:50.774680 9834 ts_manager.cc:194] Registered new tserver with Master: a690268c10b748f7995fed5d1bd9a483 (127.31.210.193:35275)
I20260808 00:32:50.774812 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:54745
I20260808 00:32:50.775852 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:43181
--local_ip_for_outbound_sockets=127.31.210.194
--tserver_master_addrs=127.31.210.254:34983
--webserver_port=38097
--webserver_interface=127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab
--principal=oryx/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:50.864661 10352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:50.864859 10352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:50.864883 10352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:50.866056 10352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:50.866102 10352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:50.866128 10352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:50.867614 10352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab
--principal=oryx/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:43181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=38097
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:50.867861 10352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:50.868136 10352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:50.873584 10366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:50.873613 10360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.873723 10352 server_base.cc:1034] running on GCE node
W20260808 00:32:50.873613 10358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:50.874145 10352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:50.874406 10352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:50.875602 10352 hybrid_clock.cc:648] HybridClock initialized: now 1786149170875576 us; error 48 us; skew 500 ppm
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:50.883095 10352 init.cc:377] Logged in from keytab as oryx/127.31.210.194@KRBTEST.COM (short username oryx)
I20260808 00:32:50.883625 10352 webserver.cc:460] Webserver started at http://127.31.210.194:38097/ using document root <none> and password file <none>
I20260808 00:32:50.883875 10352 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:50.883965 10352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:50.885628 10352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:50.886335 10374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.886555 10352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:50.886660 10352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "735b28ad727d446189a4e674982d1a1e"
format_stamp: "Formatted at 2026-08-08 00:32:49 on dist-test-slave-44d2"
server_key: "eb1b406f09299d02a3428611e77d8f59"
server_key_iv: "a488450be0ab1a2dedda54aaab3cdb06"
server_key_version: "encryptionkey@0"
I20260808 00:32:50.887038 10352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:50.916970 10352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:50.920188 10352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:50.920446 10352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:50.920831 10352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:50.921300 10352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:50.921366 10352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.921419 10352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:50.921445 10352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:50.929908 10352 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:43181
I20260808 00:32:50.929946 10533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:43181 every 8 connection(s)
I20260808 00:32:50.930410 10352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:50.932363 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 10352
I20260808 00:32:50.932585 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 9473
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.252@KRBTEST.COM
I20260808 00:32:50.946244 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.931359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:58369 (local address 127.31.210.252:38835)
0808 00:32:50.931430 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.931433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.931707 (+ 274us) server_negotiation.cc:407] Connection header received
0808 00:32:50.933377 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.933379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.933397 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.933437 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.934428 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.935195 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.936635 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.936963 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.937005 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.940586 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.940588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.940589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.940599 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.943916 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.944820 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.944822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.944822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.944899 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.945251 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.945252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.945252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.945422 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.945486 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.945937 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.946073 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:32:50.946992 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.931332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.31.210.194:58369)
0808 00:32:50.931557 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.931607 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.933249 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.933623 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.933627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.934038 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.934320 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.934325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.935309 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.935310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.936471 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.936474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.937096 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.937097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.937249 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.938023 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.938035 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.940393 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.944096 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.944108 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.944112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.944710 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.944996 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.944997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.944998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.945165 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.945526 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.945534 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.945743 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.946422 (+ 679us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.946643 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260808 00:32:50.948024 10534 heartbeater.cc:347] Connected to a master server at 127.31.210.252:38835
I20260808 00:32:50.948158 10534 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:50.948426 10534 heartbeater.cc:511] Master 127.31.210.252:38835 requested a full tablet report, sending...
I20260808 00:32:50.949008 10089 ts_manager.cc:194] Registered new tserver with Master: 735b28ad727d446189a4e674982d1a1e (127.31.210.194:43181)
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:50.950552 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:45575
--local_ip_for_outbound_sockets=127.31.210.195
--tserver_master_addrs=127.31.210.254:34983
--webserver_port=32841
--webserver_interface=127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab
--principal=oryx/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:32:50 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149170, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.253@KRBTEST.COM
I20260808 00:32:50.957602 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.931650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:59743 (local address 127.31.210.254:34983)
0808 00:32:50.931769 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.931772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.931936 (+ 164us) server_negotiation.cc:407] Connection header received
0808 00:32:50.944035 (+ 12099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.944038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.944054 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.944090 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.944869 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.945801 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.947162 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.947599 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.947654 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.952855 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.952857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.952858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.952866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.955041 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.955745 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.955747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.955747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.955818 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.956466 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.956467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.956468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.957009 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.957086 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.957329 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.957448 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:32:50.958087 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.931334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:59743)
0808 00:32:50.931858 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.931871 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.943904 (+ 12033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.944171 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.944172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.944600 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.944754 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.944757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.945949 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.945950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.947045 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.947048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.947823 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.947824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.947907 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.948454 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.948460 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.952725 (+ 4265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.955202 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.955203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.955204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.955625 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.955902 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.955903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.955904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.955977 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.957131 (+ 1154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.957132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.957186 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.957756 (+ 570us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.957892 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260808 00:32:50.958500 10535 heartbeater.cc:347] Connected to a master server at 127.31.210.254:34983
I20260808 00:32:50.958563 10535 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:50.958657 10535 heartbeater.cc:511] Master 127.31.210.254:34983 requested a full tablet report, sending...
I20260808 00:32:50.958776 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.932042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:44311 (local address 127.31.210.253:45967)
0808 00:32:50.932097 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:50.932100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:50.944276 (+ 12176us) server_negotiation.cc:407] Connection header received
0808 00:32:50.944312 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:50.944314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:50.944328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:50.944404 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:50.950678 (+ 6274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.951517 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.952758 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.953116 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.953208 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.955673 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:50.955675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:50.955675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:50.955683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:50.957384 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.957840 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.957842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.957842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.957908 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:50.958146 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:50.958150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:50.958150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:50.958272 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:50.958360 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:50.958563 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 00:32:50.958660 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:32:50.959009 9008 ts_manager.cc:194] Re-registered known tserver with Master: 735b28ad727d446189a4e674982d1a1e (127.31.210.194:43181)
I20260808 00:32:50.959136 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:50.931793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.31.210.194:44311)
0808 00:32:50.943107 (+ 11314us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:50.943123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:50.943205 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:50.950269 (+ 7064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:50.950270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:50.950484 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:50.950545 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.950547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.951634 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.951635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.952649 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:50.952652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:50.953206 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:50.953207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:50.953291 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:50.953720 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:50.953724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:50.955565 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:50.957491 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.957492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.957493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.957731 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.957984 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:50.957984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:50.957985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:50.958056 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:50.958350 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:50.958351 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:50.958401 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:32:50.958871 (+ 470us) client_negotiation.cc:240] Negotiation successful
0808 00:32:50.959000 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11286,"thread_start_us":123,"threads_started":1}
I20260808 00:32:50.959614 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:59743
I20260808 00:32:50.960067 10536 heartbeater.cc:347] Connected to a master server at 127.31.210.253:45967
I20260808 00:32:50.960139 10536 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:50.960219 10536 heartbeater.cc:511] Master 127.31.210.253:45967 requested a full tablet report, sending...
I20260808 00:32:50.960592 9834 ts_manager.cc:194] Registered new tserver with Master: 735b28ad727d446189a4e674982d1a1e (127.31.210.194:43181)
W20260808 00:32:51.048990 10544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:51.049224 10544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:51.049252 10544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:51.050632 10544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:51.050694 10544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:51.050725 10544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:51.052399 10544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab
--principal=oryx/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:45575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=32841
--tserver_master_addrs=127.31.210.254:34983,127.31.210.253:45967,127.31.210.252:38835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:51.052655 10544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:51.052889 10544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:51.058812 10555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:51.058844 10553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:51.059044 10544 server_base.cc:1034] running on GCE node
W20260808 00:32:51.058912 10557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:51.059530 10544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:51.059813 10544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:51.061038 10544 hybrid_clock.cc:648] HybridClock initialized: now 1786149171061038 us; error 44 us; skew 500 ppm
Aug 08 00:32:51 dist-test-slave-44d2 krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149171, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:51.063833 10544 init.cc:377] Logged in from keytab as oryx/127.31.210.195@KRBTEST.COM (short username oryx)
I20260808 00:32:51.064360 10544 webserver.cc:460] Webserver started at http://127.31.210.195:32841/ using document root <none> and password file <none>
I20260808 00:32:51.064671 10544 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:51.064755 10544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:51.066238 10544 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:51.066954 10564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:51.067170 10544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:51.067279 10544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "fa68f19a095045d7a18ec4bae119ba01"
format_stamp: "Formatted at 2026-08-08 00:32:50 on dist-test-slave-44d2"
server_key: "b89255722bfe8d2414d363058066ed0a"
server_key_iv: "c3767ff91b744a6365877dfc48ee70ee"
server_key_version: "encryptionkey@0"
I20260808 00:32:51.067624 10544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:51.083329 10544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:51.086153 10544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:51.086325 10544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:51.086577 10544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:32:51.086901 10544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:51.086966 10544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:51.087029 10544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:51.087061 10544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:51.094316 10544 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:45575
I20260808 00:32:51.094362 10718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:45575 every 8 connection(s)
I20260808 00:32:51.094726 10544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:51.098364 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 10544
Aug 08 00:32:51 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149171, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
Aug 08 00:32:51 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149171, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.252@KRBTEST.COM
Aug 08 00:32:51 dist-test-slave-44d2 krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149171, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.253@KRBTEST.COM
I20260808 00:32:51.111513 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.099454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.31.210.254:34983)
0808 00:32:51.099612 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.099614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.100960 (+ 1346us) server_negotiation.cc:407] Connection header received
0808 00:32:51.100985 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.100986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.100998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.101053 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.102669 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.103429 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.105013 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.105340 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.105412 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.107007 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.107008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.107008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.107015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.109989 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.110434 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.110435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.110435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.110485 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.110706 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.110707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.110708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.110849 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.110947 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.111297 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.111387 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260808 00:32:51.111809 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.095877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:37291 (local address 127.31.210.254:34983)
0808 00:32:51.095927 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.095929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.096148 (+ 219us) server_negotiation.cc:407] Connection header received
0808 00:32:51.097543 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.097544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.097558 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.097616 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.098352 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.099099 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.100882 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.101183 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.101228 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.104748 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.104750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.104750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.104758 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.109795 (+ 5037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.110405 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.110407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.110407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.110469 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.110774 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.110775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.110776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.110872 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.110917 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.111619 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.111696 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:32:51.112438 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.095550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:37291)
0808 00:32:51.096092 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.096103 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.097204 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.097633 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.097634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.097940 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:51.098194 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.098201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.099247 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.099248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.100763 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.100766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.101324 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.101324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.101480 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.102495 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:51.102500 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:51.104630 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:51.110093 (+ 5463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:51.110094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:51.110094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:51.110310 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:51.110542 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:51.110543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:51.110543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:51.110696 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:51.110950 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:51.110958 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:51.111153 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.111951 (+ 798us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.112168 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":153,"thread_start_us":103,"threads_started":1}
I20260808 00:32:51.112861 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.095551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.31.210.195:46441)
0808 00:32:51.096190 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.096198 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.097216 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.097505 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.097506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.097951 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:51.098194 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.098201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.099300 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.099301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.100765 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.100769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.101326 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.101327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.101481 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.102679 (+ 1198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:51.102684 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:51.105446 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:51.111271 (+ 5825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:51.111272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:51.111272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:51.111482 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:51.111749 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:51.111750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:51.111751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:51.111812 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:51.112147 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:51.112148 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:51.112186 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.112346 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.112449 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":137,"thread_start_us":98,"threads_started":1}
I20260808 00:32:51.113094 10720 heartbeater.cc:347] Connected to a master server at 127.31.210.254:34983
I20260808 00:32:51.113225 10720 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:51.113483 10720 heartbeater.cc:511] Master 127.31.210.254:34983 requested a full tablet report, sending...
I20260808 00:32:51.113623 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.095551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.31.210.195:56175)
0808 00:32:51.095899 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.095947 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.097203 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.097490 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.097494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.097958 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:51.098194 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.098202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.099302 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.099303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.100763 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.100766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.101326 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.101326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.101480 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.102349 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:51.102360 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:51.106140 (+ 3780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:51.109407 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:51.109417 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:51.109420 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:51.109829 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:51.112842 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:51.112843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:51.112844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:51.112914 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:51.113298 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:51.113298 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:51.113330 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.113404 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.113514 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":105,"thread_start_us":95,"threads_started":1}
I20260808 00:32:51.113620 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.095986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:56175 (local address 127.31.210.253:45967)
0808 00:32:51.096034 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.096036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.096046 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:51.097322 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.097324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.097338 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.097393 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.098321 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.099090 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.100882 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.101187 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.101228 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.106246 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.106248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.106249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.106256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.109300 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.109935 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.109936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.109936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.110001 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.113091 (+ 3090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.113093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.113093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.113225 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.113267 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.113445 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.113521 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:32:51.112583 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.095819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:46441 (local address 127.31.210.252:38835)
0808 00:32:51.095902 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.095905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.096254 (+ 349us) server_negotiation.cc:407] Connection header received
0808 00:32:51.097408 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.097409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.097422 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.097477 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.098322 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.099183 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.100883 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.101190 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.101232 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.105541 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.105543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.105543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.105551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.107679 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.111573 (+ 3894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.111574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.111575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.111656 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.111909 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.111911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.111912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.112052 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.112109 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.112303 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.112421 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:32:51.114293 9008 ts_manager.cc:194] Re-registered known tserver with Master: fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575)
I20260808 00:32:51.114326 10719 heartbeater.cc:347] Connected to a master server at 127.31.210.252:38835
I20260808 00:32:51.114377 10719 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:51.114437 10719 heartbeater.cc:511] Master 127.31.210.252:38835 requested a full tablet report, sending...
I20260808 00:32:51.114717 10089 ts_manager.cc:194] Registered new tserver with Master: fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575)
I20260808 00:32:51.114828 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:37291
I20260808 00:32:51.114961 10722 heartbeater.cc:347] Connected to a master server at 127.31.210.253:45967
I20260808 00:32:51.115027 10722 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:51.115124 10722 heartbeater.cc:511] Master 127.31.210.253:45967 requested a full tablet report, sending...
I20260808 00:32:51.115496 9834 ts_manager.cc:194] Registered new tserver with Master: fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575)
I20260808 00:32:51.117141 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.099939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.31.210.252:38835)
0808 00:32:51.100768 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.100770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.101851 (+ 1081us) server_negotiation.cc:407] Connection header received
0808 00:32:51.101894 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.101894 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.101910 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.102066 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.102656 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.103357 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.107743 (+ 4386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.108062 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.108126 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.110280 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.110282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.110283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.110291 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.115447 (+ 5156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.115988 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.115989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.115989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.116043 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.116355 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.116356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.116356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.116535 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.116633 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.116954 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.117043 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":59,"threads_started":1}
I20260808 00:32:51.116600 9008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:51.119213 9008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:51.119333 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.100533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.31.210.253:45967)
0808 00:32:51.103555 (+ 3022us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.103556 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.103713 (+ 157us) server_negotiation.cc:407] Connection header received
0808 00:32:51.103735 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.103736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.103747 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.103805 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.105702 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.106495 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.108237 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.108722 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.108838 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.111172 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.111173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.111174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.111182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.117927 (+ 6745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.118306 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.118307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.118308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.118373 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.118564 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.118565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.118565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.118700 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.118797 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.118892 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.118980 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3001,"thread_start_us":57,"threads_started":1}
I20260808 00:32:51.127663 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.123934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:35275 (local address 127.0.0.1:42732)
0808 00:32:51.124356 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.124365 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.124417 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.124636 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.124637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.124640 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:51.124712 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.124714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.125750 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.125752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.126660 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.126662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.127411 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.127412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.127472 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.127477 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.127537 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.127584 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":73,"threads_started":1}
I20260808 00:32:51.127672 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.124096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.31.210.193:35275)
0808 00:32:51.124441 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.124445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.124455 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:51.124525 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.124528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.124546 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.124634 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:51.124800 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.125654 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.126747 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.127296 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.127410 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.127472 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.127552 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.127591 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":86,"threads_started":1}
I20260808 00:32:51.128386 10251 tablet_service.cc:1467] Processing CreateTablet for tablet e9a2e1a72546435b893ccf0e986137b3 (DEFAULT_TABLE table=test-table [id=9832a8334ea446d7abc9cd22b9cb6aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:51.128839 10251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9a2e1a72546435b893ccf0e986137b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:51.128955 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.123697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45575 (local address 127.0.0.1:51016)
0808 00:32:51.124242 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.124252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.124308 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.124707 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.124708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.124709 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:51.124742 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.124744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.126318 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.126319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.127594 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.127597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.128722 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.128723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.128790 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.128794 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.128827 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.128870 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":180,"threads_started":1}
I20260808 00:32:51.129344 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.123804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.31.210.195:45575)
0808 00:32:51.124551 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.124555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.124563 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:51.124601 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.124603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.124618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.124693 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:51.124838 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.126204 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.127705 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.128600 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.129117 (+ 517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.129186 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.129216 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.129259 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":117,"threads_started":1}
I20260808 00:32:51.129882 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.123697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:43181 (local address 127.0.0.1:58318)
0808 00:32:51.124240 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.124252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.124752 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.125011 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.125012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.125014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:51.125058 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.125059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.126788 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.126789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.128315 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.128318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.129656 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.129657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.129728 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.129732 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.129795 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.129827 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260808 00:32:51.129959 10758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.123822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.31.210.194:43181)
0808 00:32:51.124185 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.124190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.124310 (+ 120us) server_negotiation.cc:407] Connection header received
0808 00:32:51.124889 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.124892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.124912 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.125015 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:51.125121 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.126685 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.128427 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.129462 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.129532 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.129567 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.129821 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.129863 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":101,"threads_started":1}
I20260808 00:32:51.132284 10761 tablet_bootstrap.cc:492] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Bootstrap starting.
I20260808 00:32:51.132741 10632 tablet_service.cc:1467] Processing CreateTablet for tablet e9a2e1a72546435b893ccf0e986137b3 (DEFAULT_TABLE table=test-table [id=9832a8334ea446d7abc9cd22b9cb6aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:51.133062 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9a2e1a72546435b893ccf0e986137b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:51.133230 10761 tablet_bootstrap.cc:654] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:51.133565 10761 log.cc:824] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:51.134466 10761 tablet_bootstrap.cc:492] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: No bootstrap required, opened a new log
I20260808 00:32:51.134593 10761 ts_tablet_manager.cc:1397] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:32:51.136157 10764 tablet_bootstrap.cc:492] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Bootstrap starting.
I20260808 00:32:51.136951 10764 tablet_bootstrap.cc:654] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:51.137092 10761 raft_consensus.cc:348] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.137151 10447 tablet_service.cc:1467] Processing CreateTablet for tablet e9a2e1a72546435b893ccf0e986137b3 (DEFAULT_TABLE table=test-table [id=9832a8334ea446d7abc9cd22b9cb6aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:51.137282 10764 log.cc:824] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:51.137287 10761 raft_consensus.cc:374] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:51.137339 10761 raft_consensus.cc:729] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a690268c10b748f7995fed5d1bd9a483, State: Initialized, Role: FOLLOWER
I20260808 00:32:51.137496 10447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9a2e1a72546435b893ccf0e986137b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:51.137467 10761 consensus_queue.cc:260] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.137804 10761 ts_tablet_manager.cc:1428] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:32:51.138252 10764 tablet_bootstrap.cc:492] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: No bootstrap required, opened a new log
I20260808 00:32:51.138320 10764 ts_tablet_manager.cc:1397] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:32:51.140456 10764 raft_consensus.cc:348] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.140641 10764 raft_consensus.cc:374] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:51.140691 10764 raft_consensus.cc:729] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa68f19a095045d7a18ec4bae119ba01, State: Initialized, Role: FOLLOWER
I20260808 00:32:51.140686 10343 heartbeater.cc:503] Master 127.31.210.254:34983 was elected leader, sending a full tablet report...
I20260808 00:32:51.140816 10764 consensus_queue.cc:260] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.141109 10764 ts_tablet_manager.cc:1428] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 00:32:51.141283 10720 heartbeater.cc:503] Master 127.31.210.254:34983 was elected leader, sending a full tablet report...
I20260808 00:32:51.141546 10769 tablet_bootstrap.cc:492] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Bootstrap starting.
I20260808 00:32:51.142274 10769 tablet_bootstrap.cc:654] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:51.142628 10769 log.cc:824] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:51.143683 10769 tablet_bootstrap.cc:492] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: No bootstrap required, opened a new log
I20260808 00:32:51.143738 10769 ts_tablet_manager.cc:1397] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:51.145891 10769 raft_consensus.cc:348] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.146087 10769 raft_consensus.cc:374] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:51.146173 10769 raft_consensus.cc:729] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735b28ad727d446189a4e674982d1a1e, State: Initialized, Role: FOLLOWER
I20260808 00:32:51.146306 10769 consensus_queue.cc:260] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.146576 10769 ts_tablet_manager.cc:1428] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:51.146677 10535 heartbeater.cc:503] Master 127.31.210.254:34983 was elected leader, sending a full tablet report...
I20260808 00:32:51.168639 10767 raft_consensus.cc:482] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:51.168843 10767 raft_consensus.cc:504] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.169360 10767 leader_election.cc:288] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 735b28ad727d446189a4e674982d1a1e (127.31.210.194:43181), fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575)
I20260808 00:32:51.171129 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.167102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.0.0.1:56042)
0808 00:32:51.167207 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.167230 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.167316 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.167568 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.167569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.167571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:51.167644 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.167646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.168427 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.168428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.169731 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.169737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.170819 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.170821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.170895 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.170899 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.170954 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.171023 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 00:32:51.171186 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.167224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.31.210.252:38835)
0808 00:32:51.167331 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.167333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.167345 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:51.167387 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.167388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.167425 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.167551 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:51.167739 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.168327 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.169859 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.170609 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.170688 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.170779 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.171029 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.171100 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 00:32:51.171602 10094 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260808 00:32:51.172667 10154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3ba4458c064fc6bc5369f6b30a6ef6. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:51.172776 10154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:32:51.174003 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.169817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:54963 (local address 127.31.210.195:45575)
0808 00:32:51.169871 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.169873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.169883 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:51.169915 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.169916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.169928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.169993 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:51.170311 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.171062 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.172903 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.173540 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.173649 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.173661 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.173867 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.173928 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:32:51.174036 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.169583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:43181 (local address 127.31.210.193:55727)
0808 00:32:51.169686 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.169697 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.169734 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.170027 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.170028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.170030 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:51.170103 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.170104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.171172 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.171173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.172751 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.172754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.173683 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.173684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.173764 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.173769 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.173840 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.173894 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":277}
I20260808 00:32:51.174080 10758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.169745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:55727 (local address 127.31.210.194:43181)
0808 00:32:51.169820 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.169822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.169831 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:51.169860 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.169860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.169871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.169947 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:51.170204 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.170985 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.172949 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.173594 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.173685 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.173695 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.173932 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.173982 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:32:51.174088 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.169583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45575 (local address 127.31.210.193:54963)
0808 00:32:51.169635 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:51.169648 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:51.169697 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:51.170094 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:51.170095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:51.170097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:51.170211 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.170213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.171321 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.171322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.172751 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:51.172754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.173661 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.173662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:51.173766 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.173769 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:51.173839 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:32:51.173894 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":6}
I20260808 00:32:51.174501 10477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a2e1a72546435b893ccf0e986137b3" candidate_uuid: "a690268c10b748f7995fed5d1bd9a483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735b28ad727d446189a4e674982d1a1e" is_pre_election: true
I20260808 00:32:51.174647 10477 raft_consensus.cc:2454] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a690268c10b748f7995fed5d1bd9a483 in term 0.
I20260808 00:32:51.174670 10659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a2e1a72546435b893ccf0e986137b3" candidate_uuid: "a690268c10b748f7995fed5d1bd9a483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa68f19a095045d7a18ec4bae119ba01" is_pre_election: true
I20260808 00:32:51.174904 10659 raft_consensus.cc:2454] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a690268c10b748f7995fed5d1bd9a483 in term 0.
I20260808 00:32:51.174959 10194 leader_election.cc:302] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 735b28ad727d446189a4e674982d1a1e, a690268c10b748f7995fed5d1bd9a483; no voters:
I20260808 00:32:51.175145 10767 raft_consensus.cc:2781] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:51.175236 10767 raft_consensus.cc:482] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:51.175256 10767 raft_consensus.cc:3037] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:51.176070 10767 raft_consensus.cc:504] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.176259 10767 leader_election.cc:288] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [CANDIDATE]: Term 1 election: Requested vote from peers 735b28ad727d446189a4e674982d1a1e (127.31.210.194:43181), fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575)
I20260808 00:32:51.176407 10477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a2e1a72546435b893ccf0e986137b3" candidate_uuid: "a690268c10b748f7995fed5d1bd9a483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735b28ad727d446189a4e674982d1a1e"
I20260808 00:32:51.176419 10659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9a2e1a72546435b893ccf0e986137b3" candidate_uuid: "a690268c10b748f7995fed5d1bd9a483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa68f19a095045d7a18ec4bae119ba01"
I20260808 00:32:51.176533 10659 raft_consensus.cc:3037] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:51.176525 10477 raft_consensus.cc:3037] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:51.177616 10477 raft_consensus.cc:2454] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a690268c10b748f7995fed5d1bd9a483 in term 1.
I20260808 00:32:51.177749 10659 raft_consensus.cc:2454] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a690268c10b748f7995fed5d1bd9a483 in term 1.
I20260808 00:32:51.177987 10191 leader_election.cc:302] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a690268c10b748f7995fed5d1bd9a483, fa68f19a095045d7a18ec4bae119ba01; no voters:
I20260808 00:32:51.178104 10767 raft_consensus.cc:2781] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:51.178313 10767 raft_consensus.cc:686] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 1 LEADER]: Becoming Leader. State: Replica: a690268c10b748f7995fed5d1bd9a483, State: Running, Role: LEADER
I20260808 00:32:51.178418 10767 consensus_queue.cc:237] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:51.179319 9008 catalog_manager.cc:5495] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 reported cstate change: term changed from 0 to 1, leader changed from <none> to a690268c10b748f7995fed5d1bd9a483 (127.31.210.193). New cstate: current_term: 1 leader_uuid: "a690268c10b748f7995fed5d1bd9a483" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } health_report { overall_health: HEALTHY } } }
I20260808 00:32:51.180033 10750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: NON_VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: true } } }
I20260808 00:32:51.180150 10750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:51.181610 10750 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: attempting to promote NON_VOTER 54d2c0b4907e42e1aec13b2b994bffa3 to VOTER
I20260808 00:32:51.181749 10750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:51.182207 10094 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 LEARNER]: Refusing update from remote peer 8c3ba4458c064fc6bc5369f6b30a6ef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260808 00:32:51.182235 9847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Refusing update from remote peer 8c3ba4458c064fc6bc5369f6b30a6ef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20260808 00:32:51.182413 10539 tablet.cc:2369] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:51.182503 10780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260808 00:32:51.182722 10750 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260808 00:32:51.183497 10750 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.183756 9847 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.183764 10094 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.184365 10750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.184439 10750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:51.184543 10750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.184587 10750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:51.184655 10750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.184697 10750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:51.184870 10154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.184907 10782 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:51.184988 10154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:32:51.185003 10782 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:32:51.188252 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.186002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.31.210.193:35275)
0808 00:32:51.186042 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.186044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.186053 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:51.186136 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.186137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.186154 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.186218 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:51.186325 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.186828 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.187590 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.187778 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.187846 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.187974 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:51.188078 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:51.188140 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.188178 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.188197 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:32:51.190402 10659 raft_consensus.cc:1261] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Refusing update from remote peer a690268c10b748f7995fed5d1bd9a483: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:51.190431 10477 raft_consensus.cc:1261] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 1 FOLLOWER]: Refusing update from remote peer a690268c10b748f7995fed5d1bd9a483: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:51.190830 10777 consensus_queue.cc:1035] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:51.191048 10767 consensus_queue.cc:1035] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [LEADER]: Connected to new peer: Peer: permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:51.194975 10787 mvcc.cc:205] Tried to move back new op lower bound from 7316067005192265728 to 7316067005147475968. Current Snapshot: MvccSnapshot[applied={T|T < 7316067005192265728}]
I20260808 00:32:51.198367 9007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:51.198491 9007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:51.200080 9007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c3ba4458c064fc6bc5369f6b30a6ef6: Sending DeleteTablet for 3 replicas of tablet e9a2e1a72546435b893ccf0e986137b3
I20260808 00:32:51.200451 10251 tablet_service.cc:1514] Processing DeleteTablet for tablet e9a2e1a72546435b893ccf0e986137b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:51 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:42732
I20260808 00:32:51.200529 10447 tablet_service.cc:1514] Processing DeleteTablet for tablet e9a2e1a72546435b893ccf0e986137b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:51 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:58318
I20260808 00:32:51.201345 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet e9a2e1a72546435b893ccf0e986137b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:51 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:51016
I20260808 00:32:51.201848 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 8964
I20260808 00:32:51.212664 10797 tablet_replica.cc:330] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: stopping tablet replica
I20260808 00:32:51.212847 10797 raft_consensus.cc:2229] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:51.213017 10797 raft_consensus.cc:2258] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:51.213733 10797 ts_tablet_manager.cc:1905] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:51.214429 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.200991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.31.210.252:38835)
0808 00:32:51.201045 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.201047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.201167 (+ 120us) server_negotiation.cc:407] Connection header received
0808 00:32:51.201274 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.201275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.201288 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.201319 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.206485 (+ 5166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.207466 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.208668 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.208993 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.209037 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.210926 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.210928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.210929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.210937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.212965 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.213443 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.213445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.213445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.213508 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.213725 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.213725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.213726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.213856 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.213931 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.214234 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.214328 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:32:51.216216 10797 ts_tablet_manager.cc:1918] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:51.216290 10797 log.cc:1196] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/e9a2e1a72546435b893ccf0e986137b3
I20260808 00:32:51.216583 10797 ts_tablet_manager.cc:1939] T e9a2e1a72546435b893ccf0e986137b3 P 735b28ad727d446189a4e674982d1a1e: Deleting consensus metadata
W20260808 00:32:51.217007 10375 connection.cc:441] server connection from 127.0.0.1:58318 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58318 (request call id 1) could send its response
I20260808 00:32:51.217597 10795 tablet_replica.cc:330] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: stopping tablet replica
I20260808 00:32:51.217739 10795 raft_consensus.cc:2229] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:51.217881 10795 raft_consensus.cc:2258] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:51.218106 10801 tablet_replica.cc:330] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: stopping tablet replica
I20260808 00:32:51.218236 10801 raft_consensus.cc:2229] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:51.218380 10801 raft_consensus.cc:2258] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:51.218925 10795 ts_tablet_manager.cc:1905] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:51.219015 10801 ts_tablet_manager.cc:1905] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:51.220891 10795 ts_tablet_manager.cc:1918] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:51.220897 10801 ts_tablet_manager.cc:1918] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:51.220984 10801 log.cc:1196] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/e9a2e1a72546435b893ccf0e986137b3
I20260808 00:32:51.221125 10795 log.cc:1196] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/e9a2e1a72546435b893ccf0e986137b3
I20260808 00:32:51.221308 10801 ts_tablet_manager.cc:1939] T e9a2e1a72546435b893ccf0e986137b3 P fa68f19a095045d7a18ec4bae119ba01: Deleting consensus metadata
I20260808 00:32:51.221553 10795 ts_tablet_manager.cc:1939] T e9a2e1a72546435b893ccf0e986137b3 P a690268c10b748f7995fed5d1bd9a483: Deleting consensus metadata
W20260808 00:32:51.221760 10568 connection.cc:441] server connection from 127.0.0.1:51016 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51016 (request call id 1) could send its response
W20260808 00:32:51.221977 10194 connection.cc:441] server connection from 127.0.0.1:42732 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42732 (request call id 1) could send its response
I20260808 00:32:51.225838 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.217278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.31.210.253:45967)
0808 00:32:51.217329 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.217331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.217341 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:51.217371 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.217372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.217388 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.217448 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.218325 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.219192 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.220314 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.220646 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.220695 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.222781 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.222782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.222782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.222791 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.224553 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.224986 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.224987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.224988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.225053 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.225281 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.225282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.225282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.225411 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.225518 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.225635 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.225724 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:32:51.226897 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:51.217371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.31.210.252:38835)
0808 00:32:51.217450 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:51.217452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:51.217958 (+ 506us) server_negotiation.cc:407] Connection header received
0808 00:32:51.218062 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:51.218063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:51.218073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:51.218101 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:51.219472 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.220308 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.221834 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:51.222159 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:51.222208 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:51.223964 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:51.223966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:51.223966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:51.223974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:51.225748 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.226158 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.226159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.226159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.226216 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:51.226430 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:51.226431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:51.226431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:51.226560 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:51.226638 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:51.226775 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 00:32:51.226836 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:32:52.196419 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.196154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:46135)
0808 00:32:52.196286 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.196370 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
W20260808 00:32:52.196777 10720 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:34983 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.197110 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.196832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:33457)
0808 00:32:52.196985 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.197064 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
W20260808 00:32:52.197410 10343 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:34983 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.197670 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.197279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:53469)
0808 00:32:52.197544 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.197620 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":59,"threads_started":1}
W20260808 00:32:52.197959 10535 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:34983 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.700881 10818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3ba4458c064fc6bc5369f6b30a6ef6)
I20260808 00:32:52.700996 10819 raft_consensus.cc:482] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3ba4458c064fc6bc5369f6b30a6ef6)
I20260808 00:32:52.701076 10818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:52.701171 10819 raft_consensus.cc:504] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:52.701515 10818 leader_election.cc:288] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983), e727e1926bcd41e6bc94090828fe4263 (127.31.210.253:45967)
I20260808 00:32:52.701519 10819 leader_election.cc:288] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983), 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252:38835)
I20260808 00:32:52.701872 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.701608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60298)
0808 00:32:52.701763 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.701827 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":62,"threads_started":1}
W20260808 00:32:52.702080 10067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.702313 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.701993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60304)
0808 00:32:52.702199 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.702265 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":61,"threads_started":1}
W20260808 00:32:52.702445 9813 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.702785 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.702697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60320)
0808 00:32:52.702736 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.702758 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 00:32:52.702893 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.702802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60330)
0808 00:32:52.702838 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.702863 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 00:32:52.702926 10067 leader_election.cc:334] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983): Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
W20260808 00:32:52.702993 9813 leader_election.cc:334] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983): Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.705619 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.701670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45967 (local address 127.0.0.1:33634)
0808 00:32:52.701845 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.701857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:52.702038 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:52.702269 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:52.702270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:52.702272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:52.702333 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:52.702335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.703316 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.703317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:52.704596 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:52.704599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.705328 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.705329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:52.705410 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:52.705416 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:32:52.705494 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 00:32:52.705544 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":59,"threads_started":1}
I20260808 00:32:52.705638 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.701829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.31.210.253:45967)
0808 00:32:52.702066 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:52.702069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:52.702077 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:32:52.702112 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:52.702113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:52.702126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:52.702204 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:52.702441 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.703217 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.704712 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.705091 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.705333 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:52.705347 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:52.705521 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 00:32:52.705571 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":67,"threads_started":1}
I20260808 00:32:52.705879 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.701730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:38835 (local address 127.0.0.1:56078)
0808 00:32:52.702128 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.702137 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:52.702223 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:52.702453 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:52.702453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:52.702468 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:52.702520 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:52.702522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.703412 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.703413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:52.704602 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:52.704605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.705585 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.705586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:52.705663 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:52.705667 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:52.705730 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:32:52.705793 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":43,"threads_started":1}
I20260808 00:32:52.705874 9847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e727e1926bcd41e6bc94090828fe4263" is_pre_election: true
I20260808 00:32:52.706022 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.701839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.31.210.252:38835)
0808 00:32:52.701990 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:52.701992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:52.702191 (+ 199us) server_negotiation.cc:407] Connection header received
0808 00:32:52.702307 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:52.702308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:52.702319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:52.702375 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:52.702621 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.703335 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.704692 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:52.705460 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:52.705585 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:52.705595 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:52.705909 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 00:32:52.705957 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":37,"threads_started":1}
I20260808 00:32:52.706048 9847 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54d2c0b4907e42e1aec13b2b994bffa3 in term 1.
I20260808 00:32:52.706219 10094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e727e1926bcd41e6bc94090828fe4263" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" is_pre_election: true
I20260808 00:32:52.706348 10094 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e727e1926bcd41e6bc94090828fe4263 in term 1.
I20260808 00:32:52.706466 10064 leader_election.cc:302] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54d2c0b4907e42e1aec13b2b994bffa3, e727e1926bcd41e6bc94090828fe4263; no voters: 8c3ba4458c064fc6bc5369f6b30a6ef6
I20260808 00:32:52.706570 9812 leader_election.cc:302] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54d2c0b4907e42e1aec13b2b994bffa3, e727e1926bcd41e6bc94090828fe4263; no voters: 8c3ba4458c064fc6bc5369f6b30a6ef6
I20260808 00:32:52.706612 10818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 00:32:52.706670 10818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c3ba4458c064fc6bc5369f6b30a6ef6)
I20260808 00:32:52.706702 10818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:32:52.706694 10819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 00:32:52.706734 10819 raft_consensus.cc:482] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c3ba4458c064fc6bc5369f6b30a6ef6)
I20260808 00:32:52.706771 10819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:32:52.707491 10818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:52.707576 10819 raft_consensus.cc:504] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:52.707659 10818 leader_election.cc:288] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983), e727e1926bcd41e6bc94090828fe4263 (127.31.210.253:45967)
I20260808 00:32:52.707738 10819 leader_election.cc:288] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983), 54d2c0b4907e42e1aec13b2b994bffa3 (127.31.210.252:38835)
I20260808 00:32:52.707854 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.707747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60340)
0808 00:32:52.707795 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.707823 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:32:52.707820 9847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e727e1926bcd41e6bc94090828fe4263"
I20260808 00:32:52.707914 9847 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 54d2c0b4907e42e1aec13b2b994bffa3 in current term 2: Already voted for candidate e727e1926bcd41e6bc94090828fe4263 in this term.
I20260808 00:32:52.707926 10094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e727e1926bcd41e6bc94090828fe4263" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54d2c0b4907e42e1aec13b2b994bffa3"
I20260808 00:32:52.708009 10094 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e727e1926bcd41e6bc94090828fe4263 in current term 2: Already voted for candidate 54d2c0b4907e42e1aec13b2b994bffa3 in this term.
I20260808 00:32:52.708168 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.708077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60348)
0808 00:32:52.708117 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.708143 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 00:32:52.708195 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.707829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60344)
0808 00:32:52.708103 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.708137 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
W20260808 00:32:52.708251 10067 leader_election.cc:334] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983): Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.708326 10067 leader_election.cc:302] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54d2c0b4907e42e1aec13b2b994bffa3; no voters: 8c3ba4458c064fc6bc5369f6b30a6ef6, e727e1926bcd41e6bc94090828fe4263
I20260808 00:32:52.708405 10818 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 00:32:52.708547 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:52.708400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60354)
0808 00:32:52.708441 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:52.708467 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 00:32:52.708652 9813 leader_election.cc:334] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983): Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:52.708709 9813 leader_election.cc:302] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e727e1926bcd41e6bc94090828fe4263; no voters: 54d2c0b4907e42e1aec13b2b994bffa3, 8c3ba4458c064fc6bc5369f6b30a6ef6
I20260808 00:32:52.708781 10819 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 00:32:53.197429 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:53.197158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:44193)
0808 00:32:53.197337 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:53.197383 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
I20260808 00:32:53.198035 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:53.197750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:42387)
0808 00:32:53.197948 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:53.197989 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260808 00:32:53.198547 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:53.198246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:60351)
0808 00:32:53.198428 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:53.198468 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260808 00:32:54.198326 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.198091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:59231)
0808 00:32:54.198234 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.198281 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260808 00:32:54.198892 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.198639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:48085)
0808 00:32:54.198792 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.198839 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260808 00:32:54.199218 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.198898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:45001)
0808 00:32:54.199115 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.199171 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":72,"threads_started":1}
I20260808 00:32:54.349673 10840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:54.349795 10840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:54.350000 10840 leader_election.cc:288] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983), e727e1926bcd41e6bc94090828fe4263 (127.31.210.253:45967)
I20260808 00:32:54.350304 9847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e727e1926bcd41e6bc94090828fe4263" is_pre_election: true
I20260808 00:32:54.350422 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.350169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60560)
0808 00:32:54.350331 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.350375 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260808 00:32:54.350481 9847 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54d2c0b4907e42e1aec13b2b994bffa3 in term 2.
I20260808 00:32:54.350729 10064 leader_election.cc:302] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54d2c0b4907e42e1aec13b2b994bffa3, e727e1926bcd41e6bc94090828fe4263; no voters:
I20260808 00:32:54.350847 10840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 00:32:54.350888 10840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:54.350934 10840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:32:54.351073 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.350800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60570)
0808 00:32:54.350991 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.351043 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260808 00:32:54.351192 10067 leader_election.cc:334] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983): Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:54.351833 10840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:54.351994 10840 leader_election.cc:288] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 3 election: Requested vote from peers 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983), e727e1926bcd41e6bc94090828fe4263 (127.31.210.253:45967)
I20260808 00:32:54.352174 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.352069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60580)
0808 00:32:54.352118 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.352145 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:32:54.352238 9847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e727e1926bcd41e6bc94090828fe4263"
I20260808 00:32:54.352326 9847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:32:54.352617 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.352468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60594)
0808 00:32:54.352552 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.352582 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 00:32:54.352728 10067 leader_election.cc:334] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983): Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
I20260808 00:32:54.353202 9847 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54d2c0b4907e42e1aec13b2b994bffa3 in term 3.
I20260808 00:32:54.353390 10064 leader_election.cc:302] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54d2c0b4907e42e1aec13b2b994bffa3, e727e1926bcd41e6bc94090828fe4263; no voters: 8c3ba4458c064fc6bc5369f6b30a6ef6
I20260808 00:32:54.353503 10840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:32:54.353624 10840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [term 3 LEADER]: Becoming Leader. State: Replica: 54d2c0b4907e42e1aec13b2b994bffa3, State: Running, Role: LEADER
I20260808 00:32:54.353716 10840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } }
I20260808 00:32:54.354130 10844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d2c0b4907e42e1aec13b2b994bffa3. Latest consensus state: current_term: 3 leader_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:54.354249 10844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:54.354490 10847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:54.355151 10847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:54.355230 10847 catalog_manager.cc:1259] Loaded cluster ID: 74dd0d7715f042418d187f293141dee1
I20260808 00:32:54.355290 10847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:32:54.355494 10847 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:54.355567 10847 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Loaded TSK: 0
I20260808 00:32:54.355737 10847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:32:54.388638 10088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:54.390434 10088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:54.391590 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.391451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60618)
0808 00:32:54.391536 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.391563 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 00:32:54.391650 9847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [term 3 FOLLOWER]: Refusing update from remote peer 54d2c0b4907e42e1aec13b2b994bffa3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 00:32:54.391898 10840 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 00:32:54.392004 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.391891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.0.0.1:60622)
0808 00:32:54.391948 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.391978 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 00:32:54.392107 10067 consensus_peers.cc:487] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 -> Peer 8c3ba4458c064fc6bc5369f6b30a6ef6 (127.31.210.254:34983): Couldn't send request to peer 8c3ba4458c064fc6bc5369f6b30a6ef6. Status: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 00:32:54.392691 10849 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d2c0b4907e42e1aec13b2b994bffa3. Latest consensus state: current_term: 3 leader_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:54.392798 10849 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:32:54.393268 10840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:54.393297 10850 mvcc.cc:205] Tried to move back new op lower bound from 7316067018305835008 to 7316067018153627648. Current Snapshot: MvccSnapshot[applied={T|T < 7316067018305835008}]
I20260808 00:32:54.393366 10840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:54.393890 10849 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3ba4458c064fc6bc5369f6b30a6ef6" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 34983 } } peers { permanent_uuid: "e727e1926bcd41e6bc94090828fe4263" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45967 } attrs { promote: false } } peers { permanent_uuid: "54d2c0b4907e42e1aec13b2b994bffa3" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 38835 } attrs { promote: false } } }
I20260808 00:32:54.394001 10849 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727e1926bcd41e6bc94090828fe4263 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:32:54.398449 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.395102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45575 (local address 127.0.0.1:51028)
0808 00:32:54.395277 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.395286 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:54.395338 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:54.395573 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:54.395574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:54.395576 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:54.395640 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.395642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.396686 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.396687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.397588 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.397590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.398225 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.398226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.398283 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.398287 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:32:54.398340 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:32:54.398378 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":21,"thread_start_us":81,"threads_started":1}
I20260808 00:32:54.398512 10860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.395272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.31.210.195:45575)
0808 00:32:54.395388 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:54.395390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:54.395399 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:54.395428 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:54.395428 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:54.395440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:54.395526 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:54.395775 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.396599 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.397675 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.398134 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.398216 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.398226 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:54.398369 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 00:32:54.398424 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260808 00:32:54.398690 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.395033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:43181 (local address 127.0.0.1:58320)
0808 00:32:54.395085 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.395096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:54.395148 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:54.395685 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:54.395686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:54.395688 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:54.395739 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.395741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.396436 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.396437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.397602 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.397604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.398485 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.398486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.398552 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.398557 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:54.398593 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 00:32:54.398635 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 00:32:54.398952 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.395111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.31.210.194:43181)
0808 00:32:54.395388 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:54.395390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:54.395399 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:54.395428 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:54.395428 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:54.395511 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:54.395806 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:54.395851 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.396343 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.397989 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.398389 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.398752 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.398761 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:54.398778 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 00:32:54.398850 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":54,"threads_started":1}
I20260808 00:32:54.399004 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 9aec1887c10c4147a4fb7975c9c1c59f (DEFAULT_TABLE table=test-table [id=1dff9d9dab264db39cb8173a9a705372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:54.399166 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aec1887c10c4147a4fb7975c9c1c59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:54.399183 10447 tablet_service.cc:1467] Processing CreateTablet for tablet 9aec1887c10c4147a4fb7975c9c1c59f (DEFAULT_TABLE table=test-table [id=1dff9d9dab264db39cb8173a9a705372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:54.399293 10447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aec1887c10c4147a4fb7975c9c1c59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:54.399340 10857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.395145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:35275 (local address 127.0.0.1:42756)
0808 00:32:54.395497 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.395508 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:54.395557 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:54.395845 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:54.395846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:54.395848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:54.395891 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.395892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.396782 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.396783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.397871 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.397874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.398971 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.398972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.399153 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.399158 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:54.399193 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 00:32:54.399234 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":98,"threads_started":1}
I20260808 00:32:54.399611 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.395371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.31.210.193:35275)
0808 00:32:54.395650 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:54.395653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:54.395660 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:32:54.395688 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:54.395689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:54.395729 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:54.395952 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:54.395991 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.396701 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.398262 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.398874 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.399473 (+ 599us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.399492 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:54.399513 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 00:32:54.399555 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":75,"threads_started":1}
I20260808 00:32:54.400799 10863 tablet_bootstrap.cc:492] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: Bootstrap starting.
I20260808 00:32:54.401114 10251 tablet_service.cc:1467] Processing CreateTablet for tablet 9aec1887c10c4147a4fb7975c9c1c59f (DEFAULT_TABLE table=test-table [id=1dff9d9dab264db39cb8173a9a705372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:54.401229 10863 tablet_bootstrap.cc:654] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:54.401242 10251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aec1887c10c4147a4fb7975c9c1c59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:54.401610 10862 tablet_bootstrap.cc:492] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Bootstrap starting.
I20260808 00:32:54.401907 10863 tablet_bootstrap.cc:492] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: No bootstrap required, opened a new log
I20260808 00:32:54.401945 10862 tablet_bootstrap.cc:654] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:54.402051 10863 ts_tablet_manager.cc:1397] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:54.402264 10863 raft_consensus.cc:348] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.402361 10863 raft_consensus.cc:374] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:54.402390 10863 raft_consensus.cc:729] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735b28ad727d446189a4e674982d1a1e, State: Initialized, Role: FOLLOWER
I20260808 00:32:54.402454 10863 consensus_queue.cc:260] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.402671 10863 ts_tablet_manager.cc:1428] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:54.402959 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.402844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:55723)
0808 00:32:54.402899 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.402933 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 00:32:54.403297 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.403212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:51299)
0808 00:32:54.403248 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.403272 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 00:32:54.402989 10862 tablet_bootstrap.cc:492] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: No bootstrap required, opened a new log
I20260808 00:32:54.403483 10862 ts_tablet_manager.cc:1397] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:54.403622 10862 raft_consensus.cc:348] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.403703 10862 raft_consensus.cc:374] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:54.403738 10862 raft_consensus.cc:729] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa68f19a095045d7a18ec4bae119ba01, State: Initialized, Role: FOLLOWER
I20260808 00:32:54.403797 10862 consensus_queue.cc:260] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.403998 10862 ts_tablet_manager.cc:1428] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:32:54.404314 10869 tablet_bootstrap.cc:492] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483: Bootstrap starting.
I20260808 00:32:54.404390 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.404184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:52483)
0808 00:32:54.404336 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.404367 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 00:32:54.404395 10719 heartbeater.cc:503] Master 127.31.210.252:38835 was elected leader, sending a full tablet report...
I20260808 00:32:54.404778 10869 tablet_bootstrap.cc:654] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:54.404781 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.404683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:43751)
0808 00:32:54.404717 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.404749 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 00:32:54.405411 10869 tablet_bootstrap.cc:492] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483: No bootstrap required, opened a new log
I20260808 00:32:54.405457 10869 ts_tablet_manager.cc:1397] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:54.405592 10869 raft_consensus.cc:348] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.405664 10869 raft_consensus.cc:374] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:54.405679 10869 raft_consensus.cc:729] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a690268c10b748f7995fed5d1bd9a483, State: Initialized, Role: FOLLOWER
I20260808 00:32:54.405730 10869 consensus_queue.cc:260] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.405889 10869 ts_tablet_manager.cc:1428] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:54.406248 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.406121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:35437)
0808 00:32:54.406179 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.406213 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:32:54.406561 10342 heartbeater.cc:503] Master 127.31.210.252:38835 was elected leader, sending a full tablet report...
I20260808 00:32:54.406647 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.406513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:59887)
0808 00:32:54.406557 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.406582 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 00:32:54.495769 10347 tablet.cc:2369] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:54.584306 10867 raft_consensus.cc:482] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:54.584422 10867 raft_consensus.cc:504] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.584834 10867 leader_election.cc:288] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575), a690268c10b748f7995fed5d1bd9a483 (127.31.210.193:35275)
I20260808 00:32:54.587909 10860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.585148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:56023 (local address 127.31.210.195:45575)
0808 00:32:54.585213 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:54.585216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:54.585384 (+ 168us) server_negotiation.cc:407] Connection header received
0808 00:32:54.585415 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:54.585416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:54.585428 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:54.585502 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:54.585700 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.586185 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.587168 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.587538 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.587624 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.587634 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:54.587794 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 00:32:54.587833 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:32:54.587941 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.585084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45575 (local address 127.31.210.194:56023)
0808 00:32:54.585282 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.585291 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:54.585350 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:54.585533 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:54.585534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:54.585536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:54.585614 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.585616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.586274 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.586275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.587066 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.587068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.587625 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.587626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.587700 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.587706 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:32:54.587775 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:32:54.587818 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":126,"threads_started":1}
I20260808 00:32:54.587937 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.585194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:44785 (local address 127.31.210.193:35275)
0808 00:32:54.585244 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:54.585246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:54.585255 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:54.585291 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:54.585292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:54.585299 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:54.585371 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:54.585713 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.586244 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.587219 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.587585 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.587675 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.587684 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:54.587826 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 00:32:54.587866 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:32:54.587945 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.585046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:35275 (local address 127.31.210.194:44785)
0808 00:32:54.585111 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.585123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:54.585178 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:54.585568 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:54.585568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:54.585571 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:54.585621 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.585622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.586340 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.586341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.587120 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:54.587122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.587677 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.587678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:54.587748 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.587751 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:54.587817 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:32:54.587857 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 00:32:54.588191 10659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aec1887c10c4147a4fb7975c9c1c59f" candidate_uuid: "735b28ad727d446189a4e674982d1a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa68f19a095045d7a18ec4bae119ba01" is_pre_election: true
I20260808 00:32:54.588330 10659 raft_consensus.cc:2454] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 735b28ad727d446189a4e674982d1a1e in term 0.
I20260808 00:32:54.588434 10279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aec1887c10c4147a4fb7975c9c1c59f" candidate_uuid: "735b28ad727d446189a4e674982d1a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a690268c10b748f7995fed5d1bd9a483" is_pre_election: true
I20260808 00:32:54.588544 10375 leader_election.cc:302] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 735b28ad727d446189a4e674982d1a1e, fa68f19a095045d7a18ec4bae119ba01; no voters:
I20260808 00:32:54.588590 10279 raft_consensus.cc:2454] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 735b28ad727d446189a4e674982d1a1e in term 0.
I20260808 00:32:54.588706 10867 raft_consensus.cc:2781] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:54.588762 10867 raft_consensus.cc:482] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:54.588799 10867 raft_consensus.cc:3037] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:54.589574 10867 raft_consensus.cc:504] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.589733 10867 leader_election.cc:288] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [CANDIDATE]: Term 1 election: Requested vote from peers fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575), a690268c10b748f7995fed5d1bd9a483 (127.31.210.193:35275)
I20260808 00:32:54.589895 10279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aec1887c10c4147a4fb7975c9c1c59f" candidate_uuid: "735b28ad727d446189a4e674982d1a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a690268c10b748f7995fed5d1bd9a483"
I20260808 00:32:54.589896 10659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aec1887c10c4147a4fb7975c9c1c59f" candidate_uuid: "735b28ad727d446189a4e674982d1a1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa68f19a095045d7a18ec4bae119ba01"
I20260808 00:32:54.589980 10279 raft_consensus.cc:3037] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:54.590036 10659 raft_consensus.cc:3037] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:54.590816 10659 raft_consensus.cc:2454] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 735b28ad727d446189a4e674982d1a1e in term 1.
I20260808 00:32:54.590866 10279 raft_consensus.cc:2454] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 735b28ad727d446189a4e674982d1a1e in term 1.
I20260808 00:32:54.591042 10379 leader_election.cc:302] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 735b28ad727d446189a4e674982d1a1e, a690268c10b748f7995fed5d1bd9a483; no voters:
I20260808 00:32:54.591132 10867 raft_consensus.cc:2781] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:54.591320 10867 raft_consensus.cc:686] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 1 LEADER]: Becoming Leader. State: Replica: 735b28ad727d446189a4e674982d1a1e, State: Running, Role: LEADER
I20260808 00:32:54.591373 10867 consensus_queue.cc:237] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } }
I20260808 00:32:54.591426 10534 heartbeater.cc:503] Master 127.31.210.252:38835 was elected leader, sending a full tablet report...
I20260808 00:32:54.592115 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.591914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:49399)
0808 00:32:54.592050 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.592087 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 00:32:54.592171 10089 catalog_manager.cc:5495] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 735b28ad727d446189a4e674982d1a1e (127.31.210.194). New cstate: current_term: 1 leader_uuid: "735b28ad727d446189a4e674982d1a1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735b28ad727d446189a4e674982d1a1e" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 } health_report { overall_health: UNKNOWN } } }
W20260808 00:32:54.598759 10723 tablet.cc:2369] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:54.626128 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.623609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58334 (local address 127.31.210.194:43181)
0808 00:32:54.623712 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:54.623714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:54.623725 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:32:54.623770 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:54.623770 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:54.623791 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:54.623882 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:54.624008 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.624564 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.625365 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:54.625569 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:54.625639 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:54.625802 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:54.625911 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:54.625976 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:54.626043 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 00:32:54.626066 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 00:32:54.627408 10659 raft_consensus.cc:1261] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Refusing update from remote peer 735b28ad727d446189a4e674982d1a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:54.627554 10279 raft_consensus.cc:1261] T 9aec1887c10c4147a4fb7975c9c1c59f P a690268c10b748f7995fed5d1bd9a483 [term 1 FOLLOWER]: Refusing update from remote peer 735b28ad727d446189a4e674982d1a1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:54.627755 10876 consensus_queue.cc:1035] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa68f19a095045d7a18ec4bae119ba01" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:54.627902 10867 consensus_queue.cc:1035] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a690268c10b748f7995fed5d1bd9a483" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:54.628829 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.628689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:54177)
0808 00:32:54.628754 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.628796 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:32:54.629295 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.629189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:50047)
0808 00:32:54.629234 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.629262 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:32:54.630685 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.630560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:56211)
0808 00:32:54.630621 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.630652 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:32:54.630685 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.630581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.195:38161)
0808 00:32:54.630624 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.630651 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:32:54.632117 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.631891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.193:55011)
0808 00:32:54.632051 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.632086 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260808 00:32:54.632167 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:54.631291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:34983 (local address 127.31.210.194:56713)
0808 00:32:54.632108 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.632139 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":800}
I20260808 00:32:54.632537 10886 mvcc.cc:205] Tried to move back new op lower bound from 7316067019271512064 to 7316067019126878208. Current Snapshot: MvccSnapshot[applied={T|T < 7316067019271512064 or (T in {7316067019271512064})}]
I20260808 00:32:54.633172 10089 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:54.633265 10089 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:54.634614 10089 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54d2c0b4907e42e1aec13b2b994bffa3: Sending DeleteTablet for 3 replicas of tablet 9aec1887c10c4147a4fb7975c9c1c59f
I20260808 00:32:54.634999 10447 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aec1887c10c4147a4fb7975c9c1c59f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:54 UTC) from {username='oryx', principal='oryx/127.31.210.252@KRBTEST.COM'} at 127.0.0.1:58320
I20260808 00:32:54.634999 10251 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aec1887c10c4147a4fb7975c9c1c59f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:54 UTC) from {username='oryx', principal='oryx/127.31.210.252@KRBTEST.COM'} at 127.0.0.1:42756
I20260808 00:32:54.635207 10897 tablet_replica.cc:330] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: stopping tablet replica
I20260808 00:32:54.635183 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aec1887c10c4147a4fb7975c9c1c59f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:54 UTC) from {username='oryx', principal='oryx/127.31.210.252@KRBTEST.COM'} at 127.0.0.1:51028
I20260808 00:32:54.635352 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 10032
I20260808 00:32:54.652798 10897 raft_consensus.cc:2229] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:54.652973 10897 raft_consensus.cc:2258] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:54.653344 10897 ts_tablet_manager.cc:1905] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:54.655141 10897 ts_tablet_manager.cc:1918] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:54.655201 10897 log.cc:1196] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/9aec1887c10c4147a4fb7975c9c1c59f
I20260808 00:32:54.655519 10897 ts_tablet_manager.cc:1939] T 9aec1887c10c4147a4fb7975c9c1c59f P 735b28ad727d446189a4e674982d1a1e: Deleting consensus metadata
I20260808 00:32:54.655874 10067 catalog_manager.cc:4841] TS 735b28ad727d446189a4e674982d1a1e (127.31.210.194:43181): tablet 9aec1887c10c4147a4fb7975c9c1c59f (table test-table [id=1dff9d9dab264db39cb8173a9a705372]) successfully deleted
I20260808 00:32:54.660772 10899 tablet_replica.cc:330] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: stopping tablet replica
I20260808 00:32:54.660848 10899 raft_consensus.cc:2229] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:54.660909 10899 raft_consensus.cc:2258] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:54.661875 10899 ts_tablet_manager.cc:1905] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:54.663727 10899 ts_tablet_manager.cc:1918] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:54.663811 10899 log.cc:1196] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/9aec1887c10c4147a4fb7975c9c1c59f
I20260808 00:32:54.664047 10899 ts_tablet_manager.cc:1939] T 9aec1887c10c4147a4fb7975c9c1c59f P fa68f19a095045d7a18ec4bae119ba01: Deleting consensus metadata
I20260808 00:32:54.664461 10064 catalog_manager.cc:4841] TS fa68f19a095045d7a18ec4bae119ba01 (127.31.210.195:45575): tablet 9aec1887c10c4147a4fb7975c9c1c59f (table test-table [id=1dff9d9dab264db39cb8173a9a705372]) successfully deleted
W20260808 00:32:54.670334 10857 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:54.670067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:35275 (local address 127.0.0.1:42766)
0808 00:32:54.670150 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:54.670163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:54.670213 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:54.670280 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:35275: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 00:32:54.670481 10067 catalog_manager.cc:4584] TS a690268c10b748f7995fed5d1bd9a483 (127.31.210.193:35275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9aec1887c10c4147a4fb7975c9c1c59f: Network error: Client connection negotiation failed: client connection to 127.31.210.193:35275: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 00:32:54.670641 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 10352
I20260808 00:32:54.684753 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 10544
I20260808 00:32:54.695134 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 9655
I20260808 00:32:54.703692 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 9905
2026-08-08T00:32:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6773 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:54 dist-test-slave-44d2 krb5kdc[10906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:54 dist-test-slave-44d2 krb5kdc[10906](info): set up 2 sockets
Aug 08 00:32:54 dist-test-slave-44d2 krb5kdc[10906](info): commencing operation
krb5kdc: starting...
W20260808 00:32:55.937747 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab.
Entry for principal oryx/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab.
I20260808 00:32:56.075579 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38703
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab
--principal=oryx/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:38703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:56.160972 10926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:56.161324 10926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:56.161365 10926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:56.162806 10926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:56.162873 10926 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:32:56.162900 10926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:56.162925 10926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:56.162955 10926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:56.162985 10926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:56.164594 10926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:38703
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.254.keytab
--principal=oryx/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:56.164881 10926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:56.165124 10926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:56.170329 10931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.170341 10935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.170295 10932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:56.170709 10926 server_base.cc:1034] running on GCE node
I20260808 00:32:56.171005 10926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:56.171329 10926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:56.172519 10926 hybrid_clock.cc:648] HybridClock initialized: now 1786149176172458 us; error 38 us; skew 500 ppm
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:56.180003 10926 init.cc:377] Logged in from keytab as oryx/127.31.210.254@KRBTEST.COM (short username oryx)
I20260808 00:32:56.180576 10926 webserver.cc:460] Webserver started at http://127.31.210.254:35229/ using document root <none> and password file <none>
I20260808 00:32:56.180847 10926 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:56.180919 10926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:56.181079 10926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:56.182206 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "55ca6afbfc774f1e857a2a2d533a835d"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "2db09236e85d13169990015f0fc7ad85"
server_key_iv: "6f6627d698f1e5d832c79b7e0df6a396"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.182601 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "55ca6afbfc774f1e857a2a2d533a835d"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "2db09236e85d13169990015f0fc7ad85"
server_key_iv: "6f6627d698f1e5d832c79b7e0df6a396"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.184114 10926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:32:56.184933 10947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.185132 10926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:56.185243 10926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "55ca6afbfc774f1e857a2a2d533a835d"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "2db09236e85d13169990015f0fc7ad85"
server_key_iv: "6f6627d698f1e5d832c79b7e0df6a396"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.185320 10926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:56.194255 10926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:56.196287 10926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:56.196468 10926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:56.201298 10926 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:38703
I20260808 00:32:56.201349 11025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:38703 every 8 connection(s)
I20260808 00:32:56.201682 10926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:32:56.202342 11026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:56.204725 11026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Bootstrap starting.
I20260808 00:32:56.205338 11026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:56.205642 11026 log.cc:824] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:56.206327 11026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: No bootstrap required, opened a new log
I20260808 00:32:56.208431 11026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ca6afbfc774f1e857a2a2d533a835d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38703 } }
I20260808 00:32:56.208662 11026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:56.208717 11026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55ca6afbfc774f1e857a2a2d533a835d, State: Initialized, Role: FOLLOWER
I20260808 00:32:56.208866 11026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ca6afbfc774f1e857a2a2d533a835d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38703 } }
I20260808 00:32:56.208951 11026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:32:56.208995 11026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:32:56.209048 11026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:56.209585 11026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ca6afbfc774f1e857a2a2d533a835d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38703 } }
I20260808 00:32:56.209707 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55ca6afbfc774f1e857a2a2d533a835d; no voters:
I20260808 00:32:56.209892 11026 leader_election.cc:288] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:32:56.209936 11030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:56.210160 11030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [term 1 LEADER]: Becoming Leader. State: Replica: 55ca6afbfc774f1e857a2a2d533a835d, State: Running, Role: LEADER
I20260808 00:32:56.210157 11026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:32:56.210266 11030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ca6afbfc774f1e857a2a2d533a835d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38703 } }
I20260808 00:32:56.210659 11031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55ca6afbfc774f1e857a2a2d533a835d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ca6afbfc774f1e857a2a2d533a835d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38703 } } }
I20260808 00:32:56.210755 11031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:56.210661 11032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ca6afbfc774f1e857a2a2d533a835d. Latest consensus state: current_term: 1 leader_uuid: "55ca6afbfc774f1e857a2a2d533a835d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ca6afbfc774f1e857a2a2d533a835d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38703 } } }
I20260808 00:32:56.210906 11032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d [sys.catalog]: This master's current role is: LEADER
I20260808 00:32:56.211308 11043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:32:56.211360 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 10926
I20260808 00:32:56.211463 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:32:56.211709 32587 external_mini_cluster.cc:1428] Setting key 079ab81cc277393cb3ba2b7525ed87af
I20260808 00:32:56.211871 11043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:32:56.217027 11043 catalog_manager.cc:1347] Generated new cluster ID: 022b87074fad4fd18b4e3f6f023646be
I20260808 00:32:56.217100 11043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:56.226816 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.212221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.31.210.254:38703)
0808 00:32:56.212390 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.212399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.212798 (+ 399us) server_negotiation.cc:407] Connection header received
0808 00:32:56.212918 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.212924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.213011 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.213166 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.213983 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.215436 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.216597 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.217363 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.217487 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.221249 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.221255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.221255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.221268 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.224963 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.225392 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.225396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.225396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.225519 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.225722 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.225722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.225723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.226030 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.226177 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.226424 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.226573 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260808 00:32:56.240321 11043 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:32:56.241063 11043 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:32:56.260689 11043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Generated new TSK 0
I20260808 00:32:56.260995 11043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab.
Entry for principal oryx/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab.
I20260808 00:32:56.303610 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38703
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab
--principal=oryx/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:56.390601 11061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:56.390813 11061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:56.390838 11061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:56.392043 11061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:56.392089 11061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:56.392109 11061 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:56.392156 11061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:32:56.393802 11061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab
--principal=oryx/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:56.394059 11061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:56.394312 11061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:56.399617 11068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.399689 11067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.399675 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:56.399809 11061 server_base.cc:1034] running on GCE node
I20260808 00:32:56.400032 11061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:56.400333 11061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:56.401517 11061 hybrid_clock.cc:648] HybridClock initialized: now 1786149176401482 us; error 64 us; skew 500 ppm
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:56.404124 11061 init.cc:377] Logged in from keytab as oryx/127.31.210.193@KRBTEST.COM (short username oryx)
I20260808 00:32:56.404726 11061 webserver.cc:460] Webserver started at http://127.31.210.193:37999/ using document root <none> and password file <none>
I20260808 00:32:56.404974 11061 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:56.405045 11061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:56.405190 11061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:56.406260 11061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "ad79792420c548e8bc9da2e9cd2fa10b"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "7538463c857e232bb75f7fe5c2563b19"
server_key_iv: "58888aed0493e8364a75b9db3a8ee55a"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.406659 11061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "ad79792420c548e8bc9da2e9cd2fa10b"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "7538463c857e232bb75f7fe5c2563b19"
server_key_iv: "58888aed0493e8364a75b9db3a8ee55a"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.408098 11061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:56.408991 11081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.409183 11061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:32:56.409296 11061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "ad79792420c548e8bc9da2e9cd2fa10b"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "7538463c857e232bb75f7fe5c2563b19"
server_key_iv: "58888aed0493e8364a75b9db3a8ee55a"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.409374 11061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:56.416142 11061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:56.418097 11061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:56.418260 11061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:56.418839 11061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:56.418906 11061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.418951 11061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:56.418972 11061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:56.427017 11061 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:46351
I20260808 00:32:56.427083 11262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:46351 every 8 connection(s)
I20260808 00:32:56.427503 11061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:32:56.429461 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.419191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:57897 (local address 127.31.210.254:38703)
0808 00:32:56.419250 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.419253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.419503 (+ 250us) server_negotiation.cc:407] Connection header received
0808 00:32:56.420097 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.420098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.420119 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.420206 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.420916 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.421504 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.422580 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.422810 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.422883 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.426178 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.426179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.426180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.426191 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.428084 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.428464 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.428465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.428465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.428575 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.428812 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.428813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.428813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.428926 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.429058 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.429312 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.429392 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:32:56.429669 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.419192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38703 (local address 127.31.210.193:57897)
0808 00:32:56.419362 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.419402 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.419984 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.420288 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.420292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.420629 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:56.420817 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.420822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.421608 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.421609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.422485 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.422487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.422907 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.422909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.423028 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.423570 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:56.423582 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:56.426011 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:56.428156 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.428161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.428162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.428386 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.428647 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.428648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.428648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.428751 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.429068 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:56.429075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:56.429196 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.429282 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.429408 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260808 00:32:56.429997 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 11061
I20260808 00:32:56.430116 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:32:56.430217 11263 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38703
I20260808 00:32:56.430327 11263 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:56.430361 32587 external_mini_cluster.cc:1428] Setting key 5f126c16af5409019d7555cfe87c1133
I20260808 00:32:56.430612 11263 heartbeater.cc:511] Master 127.31.210.254:38703 requested a full tablet report, sending...
I20260808 00:32:56.431246 10965 ts_manager.cc:194] Registered new tserver with Master: ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351)
I20260808 00:32:56.432123 10965 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:57897
I20260808 00:32:56.437485 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.431365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:39665 (local address 127.31.210.254:38703)
0808 00:32:56.431407 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.431409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.431562 (+ 153us) server_negotiation.cc:407] Connection header received
0808 00:32:56.431635 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.431635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.431645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.431685 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.432041 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.432758 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.433605 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.433787 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.433844 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.434876 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.434878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.434878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.434884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.436188 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.436647 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.436648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.436648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.436700 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.436949 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.436950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.436951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.437070 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.437142 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.437312 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.437393 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:32:56.437500 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.431353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38703 (local address 127.31.210.193:39665)
0808 00:32:56.431487 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.431497 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.431576 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.431698 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.431699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.431903 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:56.431962 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.431964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.432874 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.432875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.433495 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.433497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.433872 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.433873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.433955 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.434362 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:56.434367 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:56.434785 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:56.436272 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.436273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.436274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.436570 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.436787 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.436788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.436788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.436876 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.437147 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:56.437148 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:56.437195 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.437287 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.437393 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab.
Entry for principal oryx/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab.
I20260808 00:32:56.463362 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38703
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab
--principal=oryx/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:56.549373 11275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:56.549605 11275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:56.549629 11275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:56.550828 11275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:56.550882 11275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:56.550904 11275 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:56.550921 11275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:32:56.552592 11275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab
--principal=oryx/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:56.552891 11275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:56.553140 11275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:56.558653 11284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.558610 11286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.558599 11282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:56.559250 11275 server_base.cc:1034] running on GCE node
I20260808 00:32:56.559516 11275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:56.559775 11275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:56.560941 11275 hybrid_clock.cc:648] HybridClock initialized: now 1786149176560923 us; error 30 us; skew 500 ppm
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:56.563549 11275 init.cc:377] Logged in from keytab as oryx/127.31.210.194@KRBTEST.COM (short username oryx)
I20260808 00:32:56.564073 11275 webserver.cc:460] Webserver started at http://127.31.210.194:39327/ using document root <none> and password file <none>
I20260808 00:32:56.564309 11275 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:56.564378 11275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:56.564570 11275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:56.565656 11275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "6570e191b9cb42c99e4bf6a256f792f1"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "7c2a59bc0cbd1d91cebdfacd61a3359a"
server_key_iv: "32aba10b4eb9413d93743d34de233d51"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.566036 11275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "6570e191b9cb42c99e4bf6a256f792f1"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "7c2a59bc0cbd1d91cebdfacd61a3359a"
server_key_iv: "32aba10b4eb9413d93743d34de233d51"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.567404 11275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:32:56.568183 11296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.568557 11275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:56.568696 11275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "6570e191b9cb42c99e4bf6a256f792f1"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "7c2a59bc0cbd1d91cebdfacd61a3359a"
server_key_iv: "32aba10b4eb9413d93743d34de233d51"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.568781 11275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:56.581868 11275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:56.583810 11275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:56.583978 11275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:56.584601 11275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:56.584664 11275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.584707 11275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:56.584758 11275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:56.592424 11275 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:41289
I20260808 00:32:56.592455 11445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:41289 every 8 connection(s)
I20260808 00:32:56.592953 11275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:32:56.594754 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.585059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:38889 (local address 127.31.210.254:38703)
0808 00:32:56.585141 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.585143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.585385 (+ 242us) server_negotiation.cc:407] Connection header received
0808 00:32:56.585956 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.585958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.585972 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.586046 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.586786 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.587330 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.588977 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.589206 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.589312 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.591653 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.591654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.591655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.591663 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.593515 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.593909 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.593910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.593910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.593965 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.594196 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.594197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.594197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.594305 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.594361 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.594621 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.594689 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:32:56.594959 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.585058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38703 (local address 127.31.210.194:38889)
0808 00:32:56.585247 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.585284 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.585844 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.586140 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.586151 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.586466 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:56.586669 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.586674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.587430 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.587431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.588878 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.588882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.589405 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.589406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.589508 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.590080 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:56.590099 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:56.591544 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:56.593621 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.593626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.593627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.593842 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.594022 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.594022 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.594022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.594135 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.594363 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:56.594368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:56.594481 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.594563 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.594717 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260808 00:32:56.595467 11447 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38703
I20260808 00:32:56.595557 11447 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:56.595762 11447 heartbeater.cc:511] Master 127.31.210.254:38703 requested a full tablet report, sending...
I20260808 00:32:56.596189 10964 ts_manager.cc:194] Registered new tserver with Master: 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289)
I20260808 00:32:56.596658 10964 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:38889
I20260808 00:32:56.599593 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 11275
I20260808 00:32:56.599728 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:32:56.599951 32587 external_mini_cluster.cc:1428] Setting key 56007396269737bbe497d0e74b891fb0
I20260808 00:32:56.603048 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.596732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38703 (local address 127.31.210.194:53849)
0808 00:32:56.596856 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.596867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.596930 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.597056 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.597057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.597264 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:56.597327 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.597329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.597947 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.597948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.598451 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.598453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.598795 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.598795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.598845 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.599226 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:56.599231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:56.599708 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:56.601637 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.601638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.601639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.601881 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.602292 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.602293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.602293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.602369 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.602647 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:56.602648 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:56.602698 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.602777 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.602895 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260808 00:32:56.603127 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.596782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:53849 (local address 127.31.210.254:38703)
0808 00:32:56.596829 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.596831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.596937 (+ 106us) server_negotiation.cc:407] Connection header received
0808 00:32:56.596964 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.596965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.596975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.597036 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.597406 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.597857 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.598523 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.598729 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.598781 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.599838 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.599840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.599840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.599849 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.601513 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.602115 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.602116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.602116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.602208 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.602440 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.602440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.602441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.602580 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.602637 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.602793 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.602996 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab.
Entry for principal oryx/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab.
I20260808 00:32:56.631702 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38703
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab
--principal=oryx/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:56.719796 11456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:56.720032 11456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:56.720082 11456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:56.721351 11456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:56.721426 11456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:56.721462 11456 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:32:56.721501 11456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:32:56.723083 11456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab
--principal=oryx/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.210.254:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:56.723364 11456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:56.723609 11456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:56.729027 11469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.729163 11466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:56.729038 11462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:56.729669 11456 server_base.cc:1034] running on GCE node
I20260808 00:32:56.729923 11456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:56.730223 11456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:56.731397 11456 hybrid_clock.cc:648] HybridClock initialized: now 1786149176731372 us; error 41 us; skew 500 ppm
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:32:56.734309 11456 init.cc:377] Logged in from keytab as oryx/127.31.210.195@KRBTEST.COM (short username oryx)
I20260808 00:32:56.734800 11456 webserver.cc:460] Webserver started at http://127.31.210.195:39613/ using document root <none> and password file <none>
I20260808 00:32:56.735081 11456 fs_manager.cc:359] Metadata directory not provided
I20260808 00:32:56.735157 11456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:32:56.735313 11456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:32:56.736455 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "90c1d4eb6bf042e2a28ce16a03659d2b"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "0604a8512e8cd09dd23333a826649ad8"
server_key_iv: "6289f94e8aec23ae7de5142c09ea1a5b"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.736899 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "90c1d4eb6bf042e2a28ce16a03659d2b"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "0604a8512e8cd09dd23333a826649ad8"
server_key_iv: "6289f94e8aec23ae7de5142c09ea1a5b"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.738368 11456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:32:56.739429 11478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.739674 11456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:56.739794 11456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "90c1d4eb6bf042e2a28ce16a03659d2b"
format_stamp: "Formatted at 2026-08-08 00:32:56 on dist-test-slave-44d2"
server_key: "0604a8512e8cd09dd23333a826649ad8"
server_key_iv: "6289f94e8aec23ae7de5142c09ea1a5b"
server_key_version: "encryptionkey@0"
I20260808 00:32:56.739872 11456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:32:56.778192 11456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:32:56.780457 11456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:32:56.780690 11456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:32:56.781320 11456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:32:56.781389 11456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.781437 11456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:32:56.781466 11456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:56.789347 11456 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:39409
I20260808 00:32:56.789796 11456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:32:56.790292 11637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:39409 every 8 connection(s)
I20260808 00:32:56.792351 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.782035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:57059 (local address 127.31.210.254:38703)
0808 00:32:56.782103 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.782105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.782311 (+ 206us) server_negotiation.cc:407] Connection header received
0808 00:32:56.782988 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.782990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.783006 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.783080 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.784037 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.784932 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.786042 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.786253 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.786337 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.789340 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.789341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.789341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.789349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.790974 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.791368 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.791369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.791370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.791427 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.791773 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.791773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.791774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.791883 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.791932 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.792213 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.792282 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:32:56.792697 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.781963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38703 (local address 127.31.210.195:57059)
0808 00:32:56.782180 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.782222 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.782810 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.783252 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.783257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.783634 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:56.783932 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.783937 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.785081 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.785083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.785921 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.785923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.786356 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.786357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.786489 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.787112 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:56.787124 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:56.789211 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:56.791065 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.791071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.791073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.791298 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.791572 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.791573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.791573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.791694 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.791945 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:56.791949 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:56.792087 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.792186 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.792324 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260808 00:32:56.793241 11638 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38703
I20260808 00:32:56.793329 11638 heartbeater.cc:464] Registering TS with master...
I20260808 00:32:56.793525 11638 heartbeater.cc:511] Master 127.31.210.254:38703 requested a full tablet report, sending...
I20260808 00:32:56.793982 10964 ts_manager.cc:194] Registered new tserver with Master: 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409)
I20260808 00:32:56.794480 10964 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:57059
I20260808 00:32:56.798859 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 11456
I20260808 00:32:56.798956 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:32:56.799194 32587 external_mini_cluster.cc:1428] Setting key 2c2e827b04a6fab7f81919820c4eb0f2
I20260808 00:32:56.800112 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:32:56.801270 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.794491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:33561 (local address 127.31.210.254:38703)
0808 00:32:56.794534 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.794536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.794691 (+ 155us) server_negotiation.cc:407] Connection header received
0808 00:32:56.794758 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.794759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.794770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.794838 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.795322 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.795848 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.797111 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.797296 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.797361 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.798275 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.798277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.798277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.798283 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.800113 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.800534 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.800535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.800535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.800599 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.800815 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.800816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.800816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.800914 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.801014 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.801147 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.801208 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:32:56.801406 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.794422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38703 (local address 127.31.210.195:33561)
0808 00:32:56.794606 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.794618 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.794690 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.794866 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.794867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.795111 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:56.795191 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.795194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.796184 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.796185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.797024 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.797027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.797374 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.797375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.797424 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.797833 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:56.797837 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:56.798195 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:56.800201 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.800202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.800202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.800417 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.800670 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:56.800671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:56.800672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:56.800748 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:56.800980 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:56.800981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:56.801037 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.801128 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.801248 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:32:56.824689 11055 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:56.818854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37336 (local address 127.31.210.254:38703)
0808 00:32:56.818921 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.818924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.818934 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:32:56.819017 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.819018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.819029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.819084 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.819666 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.820200 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.821073 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.821273 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.821333 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.824572 (+ 3239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37336: BlockingRecv error: recv got EOF from 127.0.0.1:37336 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 00:32:56 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:56.833086 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.825514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37346 (local address 127.31.210.254:38703)
0808 00:32:56.825563 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.825565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.825594 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:32:56.825674 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.825675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.825683 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.825730 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:56.826410 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.826908 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.827644 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.827837 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.827978 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.830297 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:56.830298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:56.830298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:56.830305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:56.831717 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.832045 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.832046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.832046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.832100 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:56.832299 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:56.832300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:56.832300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:56.832405 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:56.832546 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.832765 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.832947 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:32:56.834899 10964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:56.836594 10964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:56.843446 11667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.839068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:39409 (local address 127.0.0.1:46186)
0808 00:32:56.839272 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.839311 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.839419 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.839966 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.839967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.839970 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:56.840041 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.840044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.841040 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.841041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.842287 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.842289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.843106 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.843107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.843191 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.843207 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.843279 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.843336 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":83,"threads_started":1}
I20260808 00:32:56.843582 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.839323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:41289 (local address 127.0.0.1:46216)
0808 00:32:56.839554 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.839562 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.839595 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.839952 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.839956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.839959 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:56.840037 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.840040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.841437 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.841438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.842448 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.842451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.843305 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.843307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.843379 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.843384 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.843457 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.843498 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":26,"thread_start_us":68,"threads_started":1}
I20260808 00:32:56.843788 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.839251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46186 (local address 127.31.210.195:39409)
0808 00:32:56.839766 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.839771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.839782 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:32:56.839826 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.839828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.839844 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.840126 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:56.840188 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.840951 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.842386 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.843027 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.843104 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.843145 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.843595 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.843639 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":102,"threads_started":1}
I20260808 00:32:56.843963 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.839579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46216 (local address 127.31.210.194:41289)
0808 00:32:56.839766 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.839771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.839783 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:32:56.839826 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.839828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.839844 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.839933 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:56.840131 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.841313 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.842520 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.843212 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.843307 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.843359 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.843795 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.843840 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260808 00:32:56.844620 11362 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe072dcd562464ba121067014bab9bf (DEFAULT_TABLE table=test-table [id=33649cae697d4b8197cf5915ae9051cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:56.844606 11546 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe072dcd562464ba121067014bab9bf (DEFAULT_TABLE table=test-table [id=33649cae697d4b8197cf5915ae9051cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:56.844911 11666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.839120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46351 (local address 127.0.0.1:52680)
0808 00:32:56.839266 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.839309 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.839379 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.840075 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.840076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.840079 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:56.840114 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.840115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.841415 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.841417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.842589 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.842592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.844678 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.844679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.844743 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.844746 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.844802 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.844840 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":30,"thread_start_us":89,"threads_started":1}
I20260808 00:32:56.844964 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.839506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.31.210.193:46351)
0808 00:32:56.839714 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.839720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.839731 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:32:56.839775 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.839777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.839795 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.840080 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:56.840183 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.841313 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.843178 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.843984 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.844698 (+ 714us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.844765 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.844818 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.844856 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260808 00:32:56.845001 11546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe072dcd562464ba121067014bab9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:56.845001 11362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe072dcd562464ba121067014bab9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:56.845631 11155 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe072dcd562464ba121067014bab9bf (DEFAULT_TABLE table=test-table [id=33649cae697d4b8197cf5915ae9051cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:56.845963 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe072dcd562464ba121067014bab9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:56.848824 11673 tablet_bootstrap.cc:492] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Bootstrap starting.
I20260808 00:32:56.849349 11675 tablet_bootstrap.cc:492] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Bootstrap starting.
I20260808 00:32:56.849573 11673 tablet_bootstrap.cc:654] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:56.849877 11673 log.cc:824] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:56.849987 11675 tablet_bootstrap.cc:654] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:56.850373 11675 log.cc:824] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:56.850713 11673 tablet_bootstrap.cc:492] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: No bootstrap required, opened a new log
I20260808 00:32:56.850797 11673 ts_tablet_manager.cc:1397] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:56.850762 11676 tablet_bootstrap.cc:492] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Bootstrap starting.
I20260808 00:32:56.851341 11676 tablet_bootstrap.cc:654] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:56.851631 11676 log.cc:824] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:32:56.852254 11675 tablet_bootstrap.cc:492] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: No bootstrap required, opened a new log
I20260808 00:32:56.852326 11675 ts_tablet_manager.cc:1397] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:32:56.852871 11676 tablet_bootstrap.cc:492] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: No bootstrap required, opened a new log
I20260808 00:32:56.852807 11673 raft_consensus.cc:348] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.852949 11676 ts_tablet_manager.cc:1397] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:56.852952 11673 raft_consensus.cc:374] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:56.852999 11673 raft_consensus.cc:729] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Initialized, Role: FOLLOWER
I20260808 00:32:56.853145 11673 consensus_queue.cc:260] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.853404 11673 ts_tablet_manager.cc:1428] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:56.853577 11263 heartbeater.cc:503] Master 127.31.210.254:38703 was elected leader, sending a full tablet report...
I20260808 00:32:56.854449 11675 raft_consensus.cc:348] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.854611 11675 raft_consensus.cc:374] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:56.854657 11675 raft_consensus.cc:729] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c1d4eb6bf042e2a28ce16a03659d2b, State: Initialized, Role: FOLLOWER
I20260808 00:32:56.854771 11675 consensus_queue.cc:260] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.855036 11675 ts_tablet_manager.cc:1428] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:56.854940 11676 raft_consensus.cc:348] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.855077 11638 heartbeater.cc:503] Master 127.31.210.254:38703 was elected leader, sending a full tablet report...
I20260808 00:32:56.855087 11676 raft_consensus.cc:374] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:56.855135 11676 raft_consensus.cc:729] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6570e191b9cb42c99e4bf6a256f792f1, State: Initialized, Role: FOLLOWER
I20260808 00:32:56.855258 11676 consensus_queue.cc:260] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.855640 11447 heartbeater.cc:503] Master 127.31.210.254:38703 was elected leader, sending a full tablet report...
I20260808 00:32:56.855682 11676 ts_tablet_manager.cc:1428] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:32:56.884581 11684 raft_consensus.cc:482] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:56.884765 11684 raft_consensus.cc:504] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.885170 11684 leader_election.cc:288] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409), 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289)
I20260808 00:32:56.888293 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.885480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:38139 (local address 127.31.210.194:41289)
0808 00:32:56.885529 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.885531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.885574 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:32:56.885608 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.885608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.885619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.885679 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:56.885871 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.886461 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.887475 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.887843 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.887900 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.887908 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.888175 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.888217 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:32:56.888309 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.885395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:41289 (local address 127.31.210.193:38139)
0808 00:32:56.885462 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.885474 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.885521 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.885704 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.885705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.885708 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:56.885790 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.885792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.886550 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.886552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.887361 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.887363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.887917 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.887918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.888073 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.888078 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.888166 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.888208 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":30}
I20260808 00:32:56.888309 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.885522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:51421 (local address 127.31.210.195:39409)
0808 00:32:56.885600 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.885608 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.885710 (+ 102us) server_negotiation.cc:407] Connection header received
0808 00:32:56.885763 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.885764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.885774 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.885839 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:56.886024 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.886494 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.887475 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.887955 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.888030 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.888038 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.888203 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.888244 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:32:56.888417 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.885395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:39409 (local address 127.31.210.193:51421)
0808 00:32:56.885628 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:56.885640 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:56.885703 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:56.885866 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:56.885867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:56.885868 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:56.885943 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.885944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.886607 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.886608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.887360 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:56.887362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.888030 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.888031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:56.888097 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.888136 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 00:32:56.888195 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:32:56.888238 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":25,"thread_start_us":128,"threads_started":1}
I20260808 00:32:56.888821 11573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe072dcd562464ba121067014bab9bf" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" is_pre_election: true
I20260808 00:32:56.888841 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe072dcd562464ba121067014bab9bf" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1" is_pre_election: true
I20260808 00:32:56.889007 11385 raft_consensus.cc:2454] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 0.
I20260808 00:32:56.889001 11573 raft_consensus.cc:2454] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 0.
I20260808 00:32:56.889256 11082 leader_election.cc:302] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c1d4eb6bf042e2a28ce16a03659d2b, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:56.889416 11684 raft_consensus.cc:2781] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:56.889497 11684 raft_consensus.cc:482] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:56.889523 11684 raft_consensus.cc:3037] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:56.890571 11684 raft_consensus.cc:504] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.890728 11684 leader_election.cc:288] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 election: Requested vote from peers 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409), 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289)
I20260808 00:32:56.890893 11573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe072dcd562464ba121067014bab9bf" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b"
I20260808 00:32:56.890913 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fe072dcd562464ba121067014bab9bf" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1"
I20260808 00:32:56.890986 11573 raft_consensus.cc:3037] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:56.890985 11385 raft_consensus.cc:3037] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:56.891954 11573 raft_consensus.cc:2454] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 1.
I20260808 00:32:56.891954 11385 raft_consensus.cc:2454] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 1.
I20260808 00:32:56.892148 11082 leader_election.cc:302] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c1d4eb6bf042e2a28ce16a03659d2b, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:56.892241 11684 raft_consensus.cc:2781] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:56.892429 11684 raft_consensus.cc:686] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 1 LEADER]: Becoming Leader. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Running, Role: LEADER
I20260808 00:32:56.892580 11684 consensus_queue.cc:237] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } }
I20260808 00:32:56.893556 10964 catalog_manager.cc:5495] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b reported cstate change: term changed from 0 to 1, leader changed from <none> to ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193). New cstate: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:56.906401 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:56.903983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52692 (local address 127.31.210.193:46351)
0808 00:32:56.904044 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:56.904046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:56.904088 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:32:56.904157 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:56.904158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:56.904174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:56.904234 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:56.904340 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.904879 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.905644 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:56.905820 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:56.905872 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:56.906042 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:56.906139 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:56.906203 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:56.906303 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 00:32:56.906321 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:32:56.908476 11385 raft_consensus.cc:1261] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Refusing update from remote peer ad79792420c548e8bc9da2e9cd2fa10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:56.908542 11573 raft_consensus.cc:1261] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Refusing update from remote peer ad79792420c548e8bc9da2e9cd2fa10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:56.908913 11689 consensus_queue.cc:1035] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:56.909129 11684 consensus_queue.cc:1035] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:56.912668 11699 mvcc.cc:205] Tried to move back new op lower bound from 7316067028613820416 to 7316067028552536064. Current Snapshot: MvccSnapshot[applied={T|T < 7316067028613820416 or (T in {7316067028613820416})}]
I20260808 00:32:56.913868 10964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:56.913981 10964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:56.914728 10964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Sending DeleteTablet for 3 replicas of tablet 4fe072dcd562464ba121067014bab9bf
I20260808 00:32:56.915064 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe072dcd562464ba121067014bab9bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:56 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52680
I20260808 00:32:56.915091 11546 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe072dcd562464ba121067014bab9bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:56 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46186
I20260808 00:32:56.915340 11711 tablet_replica.cc:330] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: stopping tablet replica
I20260808 00:32:56.915331 11362 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fe072dcd562464ba121067014bab9bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:56 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46216
I20260808 00:32:56.915439 11711 raft_consensus.cc:2229] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:56.915360 10964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:56.915518 10964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:56.915560 11711 raft_consensus.cc:2258] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:56.916086 11713 tablet_replica.cc:330] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: stopping tablet replica
I20260808 00:32:56.916262 11713 raft_consensus.cc:2229] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:56.916450 11713 raft_consensus.cc:2258] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:56.916978 11713 ts_tablet_manager.cc:1905] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:56.917105 11546 tablet_service.cc:1467] Processing CreateTablet for tablet 4b44fa54045845bcb7047d3fdf99c438 (DEFAULT_TABLE table=test-table [id=2e0989464f6848b09f1ef61eceee521e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:56.917130 11155 tablet_service.cc:1467] Processing CreateTablet for tablet 4b44fa54045845bcb7047d3fdf99c438 (DEFAULT_TABLE table=test-table [id=2e0989464f6848b09f1ef61eceee521e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:56.917253 11546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b44fa54045845bcb7047d3fdf99c438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:56.917260 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b44fa54045845bcb7047d3fdf99c438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:56.918725 11673 tablet_bootstrap.cc:492] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Bootstrap starting.
I20260808 00:32:56.918798 11362 tablet_service.cc:1467] Processing CreateTablet for tablet 4b44fa54045845bcb7047d3fdf99c438 (DEFAULT_TABLE table=test-table [id=2e0989464f6848b09f1ef61eceee521e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:56.918998 11710 tablet_replica.cc:330] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: stopping tablet replica
I20260808 00:32:56.919184 11711 ts_tablet_manager.cc:1905] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:56.919193 11673 tablet_bootstrap.cc:654] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:56.919346 11675 tablet_bootstrap.cc:492] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Bootstrap starting.
I20260808 00:32:56.919176 11710 raft_consensus.cc:2229] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:56.919785 11675 tablet_bootstrap.cc:654] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:56.920042 11710 raft_consensus.cc:2258] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:56.920400 11675 tablet_bootstrap.cc:492] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: No bootstrap required, opened a new log
I20260808 00:32:56.920459 11675 ts_tablet_manager.cc:1397] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:56.920656 11710 ts_tablet_manager.cc:1905] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:56.920667 11675 raft_consensus.cc:348] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:56.920758 11673 tablet_bootstrap.cc:492] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: No bootstrap required, opened a new log
I20260808 00:32:56.920743 11675 raft_consensus.cc:374] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:56.920799 11673 ts_tablet_manager.cc:1397] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:32:56.920804 11675 raft_consensus.cc:729] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c1d4eb6bf042e2a28ce16a03659d2b, State: Initialized, Role: FOLLOWER
I20260808 00:32:56.920885 11675 consensus_queue.cc:260] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:56.920949 11673 raft_consensus.cc:348] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:56.921005 11675 ts_tablet_manager.cc:1428] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.921017 11673 raft_consensus.cc:374] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:56.921044 11673 raft_consensus.cc:729] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Initialized, Role: FOLLOWER
I20260808 00:32:56.921092 11673 consensus_queue.cc:260] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:56.921190 11673 ts_tablet_manager.cc:1428] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:56.921185 11713 ts_tablet_manager.cc:1918] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:56.921241 11362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b44fa54045845bcb7047d3fdf99c438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:56.921276 11713 log.cc:1196] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/4fe072dcd562464ba121067014bab9bf
I20260808 00:32:56.921556 11713 ts_tablet_manager.cc:1939] T 4fe072dcd562464ba121067014bab9bf P 6570e191b9cb42c99e4bf6a256f792f1: Deleting consensus metadata
I20260808 00:32:56.920738 11711 ts_tablet_manager.cc:1918] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:56.922020 11711 log.cc:1196] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/4fe072dcd562464ba121067014bab9bf
I20260808 00:32:56.922302 11711 ts_tablet_manager.cc:1939] T 4fe072dcd562464ba121067014bab9bf P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting consensus metadata
I20260808 00:32:56.922500 11676 tablet_bootstrap.cc:492] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Bootstrap starting.
I20260808 00:32:56.922642 10952 catalog_manager.cc:4841] TS 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289): tablet 4fe072dcd562464ba121067014bab9bf (table test-table [id=33649cae697d4b8197cf5915ae9051cb]) successfully deleted
I20260808 00:32:56.922751 11710 ts_tablet_manager.cc:1918] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:56.922773 10948 catalog_manager.cc:4841] TS 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409): tablet 4fe072dcd562464ba121067014bab9bf (table test-table [id=33649cae697d4b8197cf5915ae9051cb]) successfully deleted
I20260808 00:32:56.922816 11710 log.cc:1196] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/4fe072dcd562464ba121067014bab9bf
I20260808 00:32:56.922950 11676 tablet_bootstrap.cc:654] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:56.923089 11710 ts_tablet_manager.cc:1939] T 4fe072dcd562464ba121067014bab9bf P ad79792420c548e8bc9da2e9cd2fa10b: Deleting consensus metadata
I20260808 00:32:56.923480 10952 catalog_manager.cc:4841] TS ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351): tablet 4fe072dcd562464ba121067014bab9bf (table test-table [id=33649cae697d4b8197cf5915ae9051cb]) successfully deleted
I20260808 00:32:56.923667 11676 tablet_bootstrap.cc:492] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: No bootstrap required, opened a new log
I20260808 00:32:56.923717 11676 ts_tablet_manager.cc:1397] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:32:56.923857 11676 raft_consensus.cc:348] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:56.923934 11676 raft_consensus.cc:374] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:56.923974 11676 raft_consensus.cc:729] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6570e191b9cb42c99e4bf6a256f792f1, State: Initialized, Role: FOLLOWER
I20260808 00:32:56.924024 11676 consensus_queue.cc:260] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:56.924121 11676 ts_tablet_manager.cc:1428] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260808 00:32:56.928249 11264 tablet.cc:2369] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:56.999521 11685 raft_consensus.cc:482] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:56.999655 11685 raft_consensus.cc:504] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.000090 11685 leader_election.cc:288] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351), 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289)
I20260808 00:32:57.004194 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.000680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:59663 (local address 127.31.210.194:41289)
0808 00:32:57.000735 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.000738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.000841 (+ 103us) server_negotiation.cc:407] Connection header received
0808 00:32:57.000922 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.000923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.000934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.001021 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:57.001258 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.001963 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.003378 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.003786 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.003886 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.003900 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.004070 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.004123 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:32:57.004221 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.000586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:41289 (local address 127.31.210.195:59663)
0808 00:32:57.000753 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:57.000764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:57.000859 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:57.001049 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:57.001058 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:57.001060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:57.001165 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.001166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.002059 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.002060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.003303 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.003306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.003906 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.003908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.003989 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.003994 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:32:57.004049 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:32:57.004101 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260808 00:32:57.004405 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.000533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:57121 (local address 127.31.210.193:46351)
0808 00:32:57.000598 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.000600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.000789 (+ 189us) server_negotiation.cc:407] Connection header received
0808 00:32:57.000822 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.000823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.000835 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.000882 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:32:57.001108 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.001713 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.003329 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.003894 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.003989 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.003998 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.004261 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.004315 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:32:57.004590 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.000371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46351 (local address 127.31.210.195:57121)
0808 00:32:57.000645 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:57.000658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:57.000729 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:57.000953 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:57.000954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:57.000956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:32:57.001035 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.001037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.001814 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.001816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.003212 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.003215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.003983 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.003984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.004087 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.004089 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:32:57.004144 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:32:57.004186 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
I20260808 00:32:57.004984 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b44fa54045845bcb7047d3fdf99c438" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1" is_pre_election: true
I20260808 00:32:57.004963 11180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b44fa54045845bcb7047d3fdf99c438" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" is_pre_election: true
I20260808 00:32:57.005124 11385 raft_consensus.cc:2454] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 0.
I20260808 00:32:57.005139 11180 raft_consensus.cc:2454] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 0.
I20260808 00:32:57.005362 11486 leader_election.cc:302] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6570e191b9cb42c99e4bf6a256f792f1, 90c1d4eb6bf042e2a28ce16a03659d2b; no voters:
I20260808 00:32:57.005508 11685 raft_consensus.cc:2781] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:57.005558 11685 raft_consensus.cc:482] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:57.005594 11685 raft_consensus.cc:3037] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.006351 11685 raft_consensus.cc:504] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.006515 11685 leader_election.cc:288] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 election: Requested vote from peers ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351), 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289)
I20260808 00:32:57.006702 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b44fa54045845bcb7047d3fdf99c438" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1"
I20260808 00:32:57.006722 11180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b44fa54045845bcb7047d3fdf99c438" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad79792420c548e8bc9da2e9cd2fa10b"
I20260808 00:32:57.006788 11385 raft_consensus.cc:3037] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.006798 11180 raft_consensus.cc:3037] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.007476 11180 raft_consensus.cc:2454] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 1.
I20260808 00:32:57.007673 11486 leader_election.cc:302] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c1d4eb6bf042e2a28ce16a03659d2b, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:57.007688 11385 raft_consensus.cc:2454] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 1.
I20260808 00:32:57.007781 11685 raft_consensus.cc:2781] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:57.007995 11685 raft_consensus.cc:686] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 LEADER]: Becoming Leader. State: Replica: 90c1d4eb6bf042e2a28ce16a03659d2b, State: Running, Role: LEADER
I20260808 00:32:57.008082 11685 consensus_queue.cc:237] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.009032 10964 catalog_manager.cc:5495] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195). New cstate: current_term: 1 leader_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } health_report { overall_health: HEALTHY } } }
W20260808 00:32:57.040544 11639 tablet.cc:2369] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149176, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.254@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:57.045986 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.037474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37362 (local address 127.31.210.254:38703)
0808 00:32:57.037557 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.037559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.037734 (+ 175us) server_negotiation.cc:407] Connection header received
0808 00:32:57.037774 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.037775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.037788 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.037869 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:57.038456 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.039122 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.040013 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.040221 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.040295 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.042859 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:57.042861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:57.042861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:57.042870 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:57.044560 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:57.044984 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:57.044986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:57.044987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:57.045051 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:57.045276 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:57.045277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:57.045278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:57.045431 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:57.045533 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.045708 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.045832 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:32:57.046058 11729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.037394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38703 (local address 127.0.0.1:37362)
0808 00:32:57.037630 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:57.037643 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:57.037726 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:57.037883 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:57.037884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:57.038299 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:32:57.038357 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.038359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.039209 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.039210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.039819 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.039821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.040334 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.040335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.040388 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.041108 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:32:57.041113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:32:57.042755 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:32:57.044659 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:57.044669 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:57.044672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:57.044897 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:57.045137 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:32:57.045138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:32:57.045138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:32:57.045207 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:32:57.045517 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:32:57.045526 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:32:57.045580 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 00:32:57.045679 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:32:57.045902 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":105,"thread_start_us":121,"threads_started":1}
I20260808 00:32:57.048328 10964 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:37362:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:32:57.048606 10964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:57.050148 11546 tablet_service.cc:1467] Processing CreateTablet for tablet cc92a3c30c8d489aacc61786bfeaabcd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7dee11114254bf69042c0fcfba670a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:57.050172 11155 tablet_service.cc:1467] Processing CreateTablet for tablet cc92a3c30c8d489aacc61786bfeaabcd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7dee11114254bf69042c0fcfba670a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:57.050176 11362 tablet_service.cc:1467] Processing CreateTablet for tablet cc92a3c30c8d489aacc61786bfeaabcd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7dee11114254bf69042c0fcfba670a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:32:57.050345 11362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc92a3c30c8d489aacc61786bfeaabcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.050345 11546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc92a3c30c8d489aacc61786bfeaabcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.050356 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc92a3c30c8d489aacc61786bfeaabcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.051884 11673 tablet_bootstrap.cc:492] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: Bootstrap starting.
I20260808 00:32:57.052021 11676 tablet_bootstrap.cc:492] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: Bootstrap starting.
I20260808 00:32:57.052204 11675 tablet_bootstrap.cc:492] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: Bootstrap starting.
I20260808 00:32:57.052261 11673 tablet_bootstrap.cc:654] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.052621 11676 tablet_bootstrap.cc:654] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.052683 11675 tablet_bootstrap.cc:654] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.053043 11673 tablet_bootstrap.cc:492] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: No bootstrap required, opened a new log
I20260808 00:32:57.053102 11673 ts_tablet_manager.cc:1397] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:57.053241 11673 raft_consensus.cc:348] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.053339 11673 raft_consensus.cc:374] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.053372 11673 raft_consensus.cc:729] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.053416 11673 consensus_queue.cc:260] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.053460 11676 tablet_bootstrap.cc:492] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: No bootstrap required, opened a new log
I20260808 00:32:57.053506 11676 ts_tablet_manager.cc:1397] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:57.053531 11673 ts_tablet_manager.cc:1428] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:57.053580 11703 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.053658 11703 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:57.053668 11676 raft_consensus.cc:348] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.053752 11676 raft_consensus.cc:374] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.053783 11676 raft_consensus.cc:729] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6570e191b9cb42c99e4bf6a256f792f1, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.053841 11676 consensus_queue.cc:260] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.053946 11676 ts_tablet_manager.cc:1428] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:57.054028 11687 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.054152 11687 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:57.054278 11675 tablet_bootstrap.cc:492] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: No bootstrap required, opened a new log
I20260808 00:32:57.054329 11675 ts_tablet_manager.cc:1397] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:32:57.054495 11675 raft_consensus.cc:348] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.054550 11675 raft_consensus.cc:374] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.054563 11675 raft_consensus.cc:729] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c1d4eb6bf042e2a28ce16a03659d2b, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.054615 11675 consensus_queue.cc:260] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.054713 11675 ts_tablet_manager.cc:1428] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:57.054757 11685 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.054826 11685 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 00:32:57.093703 11448 tablet.cc:2369] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:32:57.290724 11703 raft_consensus.cc:482] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:57.290858 11703 raft_consensus.cc:504] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.291289 11703 leader_election.cc:288] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289), 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409)
I20260808 00:32:57.291569 11573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc92a3c30c8d489aacc61786bfeaabcd" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" is_pre_election: true
I20260808 00:32:57.291575 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc92a3c30c8d489aacc61786bfeaabcd" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1" is_pre_election: true
I20260808 00:32:57.291692 11385 raft_consensus.cc:2454] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 0.
I20260808 00:32:57.291693 11573 raft_consensus.cc:2454] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 0.
I20260808 00:32:57.291870 11082 leader_election.cc:302] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c1d4eb6bf042e2a28ce16a03659d2b, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:57.292021 11703 raft_consensus.cc:2781] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:57.292071 11703 raft_consensus.cc:482] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:57.292099 11703 raft_consensus.cc:3037] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.292984 11703 raft_consensus.cc:504] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.293193 11703 leader_election.cc:288] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 election: Requested vote from peers 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289), 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409)
I20260808 00:32:57.293388 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc92a3c30c8d489aacc61786bfeaabcd" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1"
I20260808 00:32:57.293428 11573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc92a3c30c8d489aacc61786bfeaabcd" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b"
I20260808 00:32:57.293505 11385 raft_consensus.cc:3037] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.293501 11573 raft_consensus.cc:3037] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.294239 11385 raft_consensus.cc:2454] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 1.
I20260808 00:32:57.294269 11573 raft_consensus.cc:2454] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 1.
I20260808 00:32:57.294441 11085 leader_election.cc:302] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6570e191b9cb42c99e4bf6a256f792f1, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:57.294579 11703 raft_consensus.cc:2781] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:57.294641 11703 raft_consensus.cc:686] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [term 1 LEADER]: Becoming Leader. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Running, Role: LEADER
I20260808 00:32:57.294688 11703 consensus_queue.cc:237] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.295022 11684 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: TxnStatusTablet state changed. Reason: New leader ad79792420c548e8bc9da2e9cd2fa10b. Latest consensus state: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.295125 11684 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:57.295300 11735 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:32:57.295440 11735 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:32:57.295436 10964 catalog_manager.cc:5495] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b reported cstate change: term changed from 0 to 1, leader changed from <none> to ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193). New cstate: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:57.339561 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.337095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.31.210.193:46351)
0808 00:32:57.337175 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.337177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.337184 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:32:57.337207 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.337213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.337222 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.337291 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:57.337454 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.337945 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.338749 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.338952 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.339031 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.339225 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.339274 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.339331 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.339451 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.339476 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:32:57.339564 11729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.336840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46351 (local address 127.0.0.1:52696)
0808 00:32:57.336963 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:57.336979 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:57.337046 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:57.337305 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:57.337306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:57.337308 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:57.337371 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.337374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.338044 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.338045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.338653 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.338655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.339032 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.339033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.339102 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.339118 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.339354 (+ 236us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.339364 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 00:32:57.339443 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:32:57.339493 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 00:32:57.341102 11385 raft_consensus.cc:1261] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Refusing update from remote peer ad79792420c548e8bc9da2e9cd2fa10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.341120 11573 raft_consensus.cc:1261] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Refusing update from remote peer ad79792420c548e8bc9da2e9cd2fa10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.341352 11703 consensus_queue.cc:1035] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.341499 11703 consensus_queue.cc:1035] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.342080 11687 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: TxnStatusTablet state changed. Reason: New leader ad79792420c548e8bc9da2e9cd2fa10b. Latest consensus state: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.342125 11685 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: TxnStatusTablet state changed. Reason: New leader ad79792420c548e8bc9da2e9cd2fa10b. Latest consensus state: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.342200 11687 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:57.342214 11685 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:57.343202 11703 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.343292 11703 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:57.343230 11689 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.343334 11689 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P ad79792420c548e8bc9da2e9cd2fa10b: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:32:57.343394 11687 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.343465 11687 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P 6570e191b9cb42c99e4bf6a256f792f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:57.343345 11697 mvcc.cc:205] Tried to move back new op lower bound from 7316067030386876416 to 7316067030199365632. Current Snapshot: MvccSnapshot[applied={T|T < 7316067030386876416}]
I20260808 00:32:57.344023 11685 tablet_replica.cc:439] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } } }
I20260808 00:32:57.344115 11685 tablet_replica.cc:442] T cc92a3c30c8d489aacc61786bfeaabcd P 90c1d4eb6bf042e2a28ce16a03659d2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:32:57.344043 11698 mvcc.cc:205] Tried to move back new op lower bound from 7316067030386876416 to 7316067030199365632. Current Snapshot: MvccSnapshot[applied={T|T < 7316067030386876416}]
I20260808 00:32:57.349220 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.346418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.31.210.195:39409)
0808 00:32:57.346488 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.346490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.346499 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:32:57.346582 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.346583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.346602 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.346671 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:57.346802 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.347405 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.348376 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.348677 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.348739 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.348895 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.349012 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.349066 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.349127 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.349145 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:32:57.355096 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.352057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:41607 (local address 127.31.210.193:46351)
0808 00:32:57.352152 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.352154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.352300 (+ 146us) server_negotiation.cc:407] Connection header received
0808 00:32:57.352355 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.352356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.352368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.352416 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:57.352558 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.353144 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.354289 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.354506 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.354568 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.354829 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.354862 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.354918 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.355010 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.355032 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:32:57.355149 11746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.351984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46351 (local address 127.31.210.195:41607)
0808 00:32:57.352219 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:57.352232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:57.352297 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:57.352419 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:57.352420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:57.352430 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:57.352494 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.352495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.353238 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.353240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.354157 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.354160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.354594 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.354595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.354676 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.354694 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.354944 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.354946 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:32:57.355008 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:32:57.355056 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":141,"threads_started":1}
I20260808 00:32:57.358194 11385 raft_consensus.cc:1261] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Refusing update from remote peer 90c1d4eb6bf042e2a28ce16a03659d2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.358223 11180 raft_consensus.cc:1261] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Refusing update from remote peer 90c1d4eb6bf042e2a28ce16a03659d2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.358497 11685 consensus_queue.cc:1035] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.358908 11719 consensus_queue.cc:1035] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.368852 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.365660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:39409 (local address 127.31.210.193:43323)
0808 00:32:57.365822 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:57.365833 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:32:57.365921 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:32:57.366057 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:32:57.366058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:32:57.366059 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:32:57.366147 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.366154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.366867 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.366868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.367714 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:32:57.367717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.368226 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.368227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:32:57.368306 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.368330 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.368589 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.368592 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:32:57.368651 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:32:57.368732 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260808 00:32:57.368813 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.365689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:43323 (local address 127.31.210.195:39409)
0808 00:32:57.365732 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.365734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.365903 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:32:57.365978 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.365979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.365990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.366053 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:57.366235 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.366767 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.367853 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.368139 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.368226 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.368445 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.368512 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.368586 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.368657 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.368724 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:32:57.378849 10964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:57.378996 10964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:57.379738 10964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Sending DeleteTablet for 3 replicas of tablet 4b44fa54045845bcb7047d3fdf99c438
I20260808 00:32:57.380089 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b44fa54045845bcb7047d3fdf99c438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52680
I20260808 00:32:57.380110 11546 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b44fa54045845bcb7047d3fdf99c438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46186
I20260808 00:32:57.380198 11710 tablet_replica.cc:330] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: stopping tablet replica
I20260808 00:32:57.380220 11711 tablet_replica.cc:330] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: stopping tablet replica
I20260808 00:32:57.380250 11710 raft_consensus.cc:2229] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:57.380277 11711 raft_consensus.cc:2229] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:57.380287 11710 raft_consensus.cc:2258] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.380363 11711 raft_consensus.cc:2258] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.380693 11710 ts_tablet_manager.cc:1905] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.380734 11711 ts_tablet_manager.cc:1905] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.382560 11362 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b44fa54045845bcb7047d3fdf99c438 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46216
I20260808 00:32:57.382707 11713 tablet_replica.cc:330] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: stopping tablet replica
I20260808 00:32:57.382767 11713 raft_consensus.cc:2229] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:57.382822 11713 raft_consensus.cc:2258] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.383235 11713 ts_tablet_manager.cc:1905] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.384598 11713 ts_tablet_manager.cc:1918] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:57.384680 11713 log.cc:1196] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/4b44fa54045845bcb7047d3fdf99c438
I20260808 00:32:57.384934 11713 ts_tablet_manager.cc:1939] T 4b44fa54045845bcb7047d3fdf99c438 P 6570e191b9cb42c99e4bf6a256f792f1: Deleting consensus metadata
I20260808 00:32:57.385308 11710 ts_tablet_manager.cc:1918] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:57.385327 10952 catalog_manager.cc:4841] TS 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289): tablet 4b44fa54045845bcb7047d3fdf99c438 (table test-table [id=2e0989464f6848b09f1ef61eceee521e]) successfully deleted
I20260808 00:32:57.385383 11710 log.cc:1196] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/4b44fa54045845bcb7047d3fdf99c438
I20260808 00:32:57.385633 11710 ts_tablet_manager.cc:1939] T 4b44fa54045845bcb7047d3fdf99c438 P ad79792420c548e8bc9da2e9cd2fa10b: Deleting consensus metadata
I20260808 00:32:57.385993 11711 ts_tablet_manager.cc:1918] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:57.386023 10952 catalog_manager.cc:4841] TS ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351): tablet 4b44fa54045845bcb7047d3fdf99c438 (table test-table [id=2e0989464f6848b09f1ef61eceee521e]) successfully deleted
I20260808 00:32:57.386060 11711 log.cc:1196] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/4b44fa54045845bcb7047d3fdf99c438
I20260808 00:32:57.386292 11711 ts_tablet_manager.cc:1939] T 4b44fa54045845bcb7047d3fdf99c438 P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting consensus metadata
I20260808 00:32:57.386667 10948 catalog_manager.cc:4841] TS 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409): tablet 4b44fa54045845bcb7047d3fdf99c438 (table test-table [id=2e0989464f6848b09f1ef61eceee521e]) successfully deleted
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[10906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:32:57.416827 11055 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:32:57.408985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37378 (local address 127.31.210.254:38703)
0808 00:32:57.409049 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.409051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.409274 (+ 223us) server_negotiation.cc:407] Connection header received
0808 00:32:57.409349 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.409350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.409364 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.409452 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:57.410347 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.411148 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.412553 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.412862 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.412937 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.416756 (+ 3819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37378: BlockingRecv error: recv got EOF from 127.0.0.1:37378 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[10906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM
I20260808 00:32:57.427775 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.418139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37392 (local address 127.31.210.254:38703)
0808 00:32:57.418179 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.418181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.418457 (+ 276us) server_negotiation.cc:407] Connection header received
0808 00:32:57.418487 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.418487 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.418498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.418558 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:32:57.419230 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.419952 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.421107 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.421370 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.421459 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.424332 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:32:57.424333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:32:57.424334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:32:57.424342 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:32:57.426256 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:57.426690 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:57.426691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:57.426691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:57.426753 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:32:57.426980 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:32:57.426981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:32:57.426982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:32:57.427237 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:32:57.427323 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.427585 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.427677 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:32:57.429623 10964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:57.429800 10964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:57.431219 11155 tablet_service.cc:1467] Processing CreateTablet for tablet ffcbffd29c064871adcfefb7b8c86a56 (DEFAULT_TABLE table=test-table [id=7545b2528e9a44a4ad3738c3af892eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:57.431212 11362 tablet_service.cc:1467] Processing CreateTablet for tablet ffcbffd29c064871adcfefb7b8c86a56 (DEFAULT_TABLE table=test-table [id=7545b2528e9a44a4ad3738c3af892eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:57.431355 11362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffcbffd29c064871adcfefb7b8c86a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.431355 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffcbffd29c064871adcfefb7b8c86a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.432775 11676 tablet_bootstrap.cc:492] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: Bootstrap starting.
I20260808 00:32:57.432842 11673 tablet_bootstrap.cc:492] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: Bootstrap starting.
I20260808 00:32:57.433212 11673 tablet_bootstrap.cc:654] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.433212 11676 tablet_bootstrap.cc:654] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.433559 11546 tablet_service.cc:1467] Processing CreateTablet for tablet ffcbffd29c064871adcfefb7b8c86a56 (DEFAULT_TABLE table=test-table [id=7545b2528e9a44a4ad3738c3af892eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:57.433699 11546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffcbffd29c064871adcfefb7b8c86a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.434032 11673 tablet_bootstrap.cc:492] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: No bootstrap required, opened a new log
I20260808 00:32:57.434087 11673 ts_tablet_manager.cc:1397] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:57.434190 11673 raft_consensus.cc:348] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.434259 11673 raft_consensus.cc:374] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.434283 11673 raft_consensus.cc:729] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.434329 11673 consensus_queue.cc:260] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.434437 11673 ts_tablet_manager.cc:1428] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:57.435145 11675 tablet_bootstrap.cc:492] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: Bootstrap starting.
I20260808 00:32:57.435609 11675 tablet_bootstrap.cc:654] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.436097 11676 tablet_bootstrap.cc:492] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: No bootstrap required, opened a new log
I20260808 00:32:57.436159 11676 ts_tablet_manager.cc:1397] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 00:32:57.436267 11675 tablet_bootstrap.cc:492] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: No bootstrap required, opened a new log
I20260808 00:32:57.436316 11675 ts_tablet_manager.cc:1397] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:57.436334 11676 raft_consensus.cc:348] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.436434 11676 raft_consensus.cc:374] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.436460 11676 raft_consensus.cc:729] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6570e191b9cb42c99e4bf6a256f792f1, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.436468 11675 raft_consensus.cc:348] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.436569 11675 raft_consensus.cc:374] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.436544 11676 consensus_queue.cc:260] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.436599 11675 raft_consensus.cc:729] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c1d4eb6bf042e2a28ce16a03659d2b, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.436647 11676 ts_tablet_manager.cc:1428] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:57.436647 11675 consensus_queue.cc:260] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.436733 11675 ts_tablet_manager.cc:1428] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:57.488806 11684 raft_consensus.cc:482] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:57.488947 11684 raft_consensus.cc:504] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.489257 11684 leader_election.cc:288] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289), 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409)
I20260808 00:32:57.489470 11573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcbffd29c064871adcfefb7b8c86a56" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" is_pre_election: true
I20260808 00:32:57.489478 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcbffd29c064871adcfefb7b8c86a56" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1" is_pre_election: true
I20260808 00:32:57.489578 11573 raft_consensus.cc:2454] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 0.
I20260808 00:32:57.489611 11385 raft_consensus.cc:2454] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 0.
I20260808 00:32:57.489760 11082 leader_election.cc:302] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c1d4eb6bf042e2a28ce16a03659d2b, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:57.489883 11684 raft_consensus.cc:2781] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:57.489928 11684 raft_consensus.cc:482] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:57.489965 11684 raft_consensus.cc:3037] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.490756 11684 raft_consensus.cc:504] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.490918 11684 leader_election.cc:288] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 election: Requested vote from peers 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289), 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409)
I20260808 00:32:57.491138 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcbffd29c064871adcfefb7b8c86a56" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1"
I20260808 00:32:57.491158 11573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffcbffd29c064871adcfefb7b8c86a56" candidate_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b"
I20260808 00:32:57.491235 11385 raft_consensus.cc:3037] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.491257 11573 raft_consensus.cc:3037] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.492059 11385 raft_consensus.cc:2454] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 1.
I20260808 00:32:57.492208 11573 raft_consensus.cc:2454] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79792420c548e8bc9da2e9cd2fa10b in term 1.
I20260808 00:32:57.492250 11085 leader_election.cc:302] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6570e191b9cb42c99e4bf6a256f792f1, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:57.492337 11684 raft_consensus.cc:2781] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:57.492400 11684 raft_consensus.cc:686] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 LEADER]: Becoming Leader. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Running, Role: LEADER
I20260808 00:32:57.492467 11684 consensus_queue.cc:237] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.493209 10964 catalog_manager.cc:5495] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b reported cstate change: term changed from 0 to 1, leader changed from <none> to ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193). New cstate: current_term: 1 leader_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } health_report { overall_health: UNKNOWN } } }
I20260808 00:32:57.498924 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.496173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.31.210.193:46351)
0808 00:32:57.496244 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.496247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.496319 (+ 72us) server_negotiation.cc:407] Connection header received
0808 00:32:57.496439 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.496442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.496459 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.496569 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:57.496729 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.497260 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.498125 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.498358 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.498453 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.498638 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.498681 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.498750 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.498819 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.498839 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:32:57.499819 11573 raft_consensus.cc:1261] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Refusing update from remote peer ad79792420c548e8bc9da2e9cd2fa10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.499825 11385 raft_consensus.cc:1261] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Refusing update from remote peer ad79792420c548e8bc9da2e9cd2fa10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.500070 11703 consensus_queue.cc:1035] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.500164 11684 consensus_queue.cc:1035] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.503662 10964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:57.503765 10964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:57.504451 10964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Sending DeleteTablet for 3 replicas of tablet ffcbffd29c064871adcfefb7b8c86a56
I20260808 00:32:57.504747 11546 tablet_service.cc:1514] Processing DeleteTablet for tablet ffcbffd29c064871adcfefb7b8c86a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46186
I20260808 00:32:57.504865 11711 tablet_replica.cc:330] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: stopping tablet replica
I20260808 00:32:57.504873 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet ffcbffd29c064871adcfefb7b8c86a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52680
I20260808 00:32:57.504927 11711 raft_consensus.cc:2229] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:57.504974 11710 tablet_replica.cc:330] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: stopping tablet replica
I20260808 00:32:57.504997 11711 raft_consensus.cc:2258] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.505013 11362 tablet_service.cc:1514] Processing DeleteTablet for tablet ffcbffd29c064871adcfefb7b8c86a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46216
I20260808 00:32:57.505039 11710 raft_consensus.cc:2229] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:57.505089 11713 tablet_replica.cc:330] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: stopping tablet replica
I20260808 00:32:57.505110 11710 raft_consensus.cc:2258] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.505136 11713 raft_consensus.cc:2229] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:57.505174 11713 raft_consensus.cc:2258] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.505316 10964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:32:57.505458 10964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:32:57.505496 11713 ts_tablet_manager.cc:1905] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.505875 11710 ts_tablet_manager.cc:1905] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.507160 11713 ts_tablet_manager.cc:1918] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:57.507227 11710 ts_tablet_manager.cc:1918] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:57.507262 11713 log.cc:1196] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/ffcbffd29c064871adcfefb7b8c86a56
I20260808 00:32:57.507278 11710 log.cc:1196] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/ffcbffd29c064871adcfefb7b8c86a56
I20260808 00:32:57.507503 11710 ts_tablet_manager.cc:1939] T ffcbffd29c064871adcfefb7b8c86a56 P ad79792420c548e8bc9da2e9cd2fa10b: Deleting consensus metadata
I20260808 00:32:57.507530 11713 ts_tablet_manager.cc:1939] T ffcbffd29c064871adcfefb7b8c86a56 P 6570e191b9cb42c99e4bf6a256f792f1: Deleting consensus metadata
I20260808 00:32:57.508083 11711 ts_tablet_manager.cc:1905] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.508592 11155 tablet_service.cc:1467] Processing CreateTablet for tablet ee37a80889ef431d9f8122a92dacaacb (DEFAULT_TABLE table=test-table [id=178e673f35014379b6afd62b1124e29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:57.508704 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee37a80889ef431d9f8122a92dacaacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.509584 11711 ts_tablet_manager.cc:1918] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:32:57.509672 11711 log.cc:1196] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/ffcbffd29c064871adcfefb7b8c86a56
I20260808 00:32:57.509824 11673 tablet_bootstrap.cc:492] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b: Bootstrap starting.
I20260808 00:32:57.509918 11711 ts_tablet_manager.cc:1939] T ffcbffd29c064871adcfefb7b8c86a56 P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting consensus metadata
I20260808 00:32:57.510111 11673 tablet_bootstrap.cc:654] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.510310 10948 catalog_manager.cc:4841] TS 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409): tablet ffcbffd29c064871adcfefb7b8c86a56 (table test-table [id=7545b2528e9a44a4ad3738c3af892eae]) successfully deleted
I20260808 00:32:57.510437 11546 tablet_service.cc:1467] Processing CreateTablet for tablet ee37a80889ef431d9f8122a92dacaacb (DEFAULT_TABLE table=test-table [id=178e673f35014379b6afd62b1124e29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:57.510545 11546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee37a80889ef431d9f8122a92dacaacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.511677 11675 tablet_bootstrap.cc:492] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: Bootstrap starting.
I20260808 00:32:57.512135 11675 tablet_bootstrap.cc:654] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.512562 10952 catalog_manager.cc:4841] TS 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289): tablet ffcbffd29c064871adcfefb7b8c86a56 (table test-table [id=7545b2528e9a44a4ad3738c3af892eae]) successfully deleted
I20260808 00:32:57.512697 11362 tablet_service.cc:1467] Processing CreateTablet for tablet ee37a80889ef431d9f8122a92dacaacb (DEFAULT_TABLE table=test-table [id=178e673f35014379b6afd62b1124e29b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:32:57.512797 11362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee37a80889ef431d9f8122a92dacaacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:32:57.512900 11675 tablet_bootstrap.cc:492] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: No bootstrap required, opened a new log
I20260808 00:32:57.512944 11675 ts_tablet_manager.cc:1397] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:57.513063 11675 raft_consensus.cc:348] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.513151 11675 raft_consensus.cc:374] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.513173 11675 raft_consensus.cc:729] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c1d4eb6bf042e2a28ce16a03659d2b, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.513238 11675 consensus_queue.cc:260] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.513331 11675 ts_tablet_manager.cc:1428] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:32:57.514091 11676 tablet_bootstrap.cc:492] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: Bootstrap starting.
I20260808 00:32:57.514264 10952 catalog_manager.cc:4841] TS ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351): tablet ffcbffd29c064871adcfefb7b8c86a56 (table test-table [id=7545b2528e9a44a4ad3738c3af892eae]) successfully deleted
I20260808 00:32:57.514541 11673 tablet_bootstrap.cc:492] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b: No bootstrap required, opened a new log
I20260808 00:32:57.514581 11673 ts_tablet_manager.cc:1397] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 00:32:57.514703 11676 tablet_bootstrap.cc:654] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: Neither blocks nor log segments found. Creating new log.
I20260808 00:32:57.514678 11673 raft_consensus.cc:348] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.514755 11673 raft_consensus.cc:374] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.514779 11673 raft_consensus.cc:729] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad79792420c548e8bc9da2e9cd2fa10b, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.514820 11673 consensus_queue.cc:260] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.514904 11673 ts_tablet_manager.cc:1428] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:57.515508 11676 tablet_bootstrap.cc:492] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: No bootstrap required, opened a new log
I20260808 00:32:57.515563 11676 ts_tablet_manager.cc:1397] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:32:57.515708 11676 raft_consensus.cc:348] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.515776 11676 raft_consensus.cc:374] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:32:57.515790 11676 raft_consensus.cc:729] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6570e191b9cb42c99e4bf6a256f792f1, State: Initialized, Role: FOLLOWER
I20260808 00:32:57.515843 11676 consensus_queue.cc:260] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.515937 11676 ts_tablet_manager.cc:1428] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:32:57.565155 11685 raft_consensus.cc:482] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:32:57.565267 11685 raft_consensus.cc:504] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.565609 11685 leader_election.cc:288] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351), 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289)
I20260808 00:32:57.565881 11180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee37a80889ef431d9f8122a92dacaacb" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" is_pre_election: true
I20260808 00:32:57.565918 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee37a80889ef431d9f8122a92dacaacb" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1" is_pre_election: true
I20260808 00:32:57.566026 11180 raft_consensus.cc:2454] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 0.
I20260808 00:32:57.566026 11385 raft_consensus.cc:2454] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 0.
I20260808 00:32:57.566221 11486 leader_election.cc:302] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c1d4eb6bf042e2a28ce16a03659d2b, ad79792420c548e8bc9da2e9cd2fa10b; no voters:
I20260808 00:32:57.566378 11685 raft_consensus.cc:2781] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:32:57.566495 11685 raft_consensus.cc:482] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:32:57.566529 11685 raft_consensus.cc:3037] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.567219 11685 raft_consensus.cc:504] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.567363 11685 leader_election.cc:288] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 election: Requested vote from peers ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351), 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289)
I20260808 00:32:57.567601 11180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee37a80889ef431d9f8122a92dacaacb" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad79792420c548e8bc9da2e9cd2fa10b"
I20260808 00:32:57.567605 11385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee37a80889ef431d9f8122a92dacaacb" candidate_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6570e191b9cb42c99e4bf6a256f792f1"
I20260808 00:32:57.567698 11180 raft_consensus.cc:3037] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.567695 11385 raft_consensus.cc:3037] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:32:57.568548 11385 raft_consensus.cc:2454] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 1.
I20260808 00:32:57.568634 11180 raft_consensus.cc:2454] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c1d4eb6bf042e2a28ce16a03659d2b in term 1.
I20260808 00:32:57.568775 11486 leader_election.cc:302] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6570e191b9cb42c99e4bf6a256f792f1, 90c1d4eb6bf042e2a28ce16a03659d2b; no voters:
I20260808 00:32:57.568874 11685 raft_consensus.cc:2781] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:32:57.568936 11685 raft_consensus.cc:686] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 LEADER]: Becoming Leader. State: Replica: 90c1d4eb6bf042e2a28ce16a03659d2b, State: Running, Role: LEADER
I20260808 00:32:57.569069 11685 consensus_queue.cc:237] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } }
I20260808 00:32:57.569718 10964 catalog_manager.cc:5495] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195). New cstate: current_term: 1 leader_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c1d4eb6bf042e2a28ce16a03659d2b" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39409 } health_report { overall_health: HEALTHY } } }
I20260808 00:32:57.591198 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.588880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.31.210.195:39409)
0808 00:32:57.588926 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:32:57.588928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:32:57.588999 (+ 71us) server_negotiation.cc:407] Connection header received
0808 00:32:57.589071 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:32:57.589072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:32:57.589084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:32:57.589130 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:32:57.589235 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.589701 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.590495 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:32:57.590674 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:32:57.590730 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:32:57.590899 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:32:57.590927 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:32:57.590968 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 00:32:57.591078 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 00:32:57.591097 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:32:57.594056 11385 raft_consensus.cc:1261] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Refusing update from remote peer 90c1d4eb6bf042e2a28ce16a03659d2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.594069 11180 raft_consensus.cc:1261] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Refusing update from remote peer 90c1d4eb6bf042e2a28ce16a03659d2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:32:57.594312 11685 consensus_queue.cc:1035] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570e191b9cb42c99e4bf6a256f792f1" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.594480 11685 consensus_queue.cc:1035] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad79792420c548e8bc9da2e9cd2fa10b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 46351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:32:57.605553 10964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:57.605659 10964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:32:57.606389 10964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55ca6afbfc774f1e857a2a2d533a835d: Sending DeleteTablet for 3 replicas of tablet ee37a80889ef431d9f8122a92dacaacb
I20260808 00:32:57.606652 11546 tablet_service.cc:1514] Processing DeleteTablet for tablet ee37a80889ef431d9f8122a92dacaacb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46186
I20260808 00:32:57.606691 11362 tablet_service.cc:1514] Processing DeleteTablet for tablet ee37a80889ef431d9f8122a92dacaacb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:46216
I20260808 00:32:57.606781 11711 tablet_replica.cc:330] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: stopping tablet replica
I20260808 00:32:57.606806 11713 tablet_replica.cc:330] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: stopping tablet replica
I20260808 00:32:57.606844 11711 raft_consensus.cc:2229] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:32:57.606853 11713 raft_consensus.cc:2229] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:57.606884 11713 raft_consensus.cc:2258] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.606907 11711 raft_consensus.cc:2258] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.607149 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet ee37a80889ef431d9f8122a92dacaacb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:32:57 UTC) from {username='oryx', principal='oryx/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52680
I20260808 00:32:57.607193 11711 ts_tablet_manager.cc:1905] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.607254 11710 tablet_replica.cc:330] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b: stopping tablet replica
I20260808 00:32:57.607324 11710 raft_consensus.cc:2229] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:32:57.607368 11710 raft_consensus.cc:2258] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:32:57.607440 11713 ts_tablet_manager.cc:1905] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.607535 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 11061
I20260808 00:32:57.607694 11710 ts_tablet_manager.cc:1905] T ee37a80889ef431d9f8122a92dacaacb P ad79792420c548e8bc9da2e9cd2fa10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:32:57.609326 11711 ts_tablet_manager.cc:1918] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:57.609393 11711 log.cc:1196] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/ee37a80889ef431d9f8122a92dacaacb
I20260808 00:32:57.609647 11711 ts_tablet_manager.cc:1939] T ee37a80889ef431d9f8122a92dacaacb P 90c1d4eb6bf042e2a28ce16a03659d2b: Deleting consensus metadata
I20260808 00:32:57.609848 11713 ts_tablet_manager.cc:1918] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:32:57.609928 11713 log.cc:1196] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/ee37a80889ef431d9f8122a92dacaacb
I20260808 00:32:57.610175 11713 ts_tablet_manager.cc:1939] T ee37a80889ef431d9f8122a92dacaacb P 6570e191b9cb42c99e4bf6a256f792f1: Deleting consensus metadata
I20260808 00:32:57.610491 10952 catalog_manager.cc:4841] TS 6570e191b9cb42c99e4bf6a256f792f1 (127.31.210.194:41289): tablet ee37a80889ef431d9f8122a92dacaacb (table test-table [id=178e673f35014379b6afd62b1124e29b]) successfully deleted
I20260808 00:32:57.610671 10948 catalog_manager.cc:4841] TS 90c1d4eb6bf042e2a28ce16a03659d2b (127.31.210.195:39409): tablet ee37a80889ef431d9f8122a92dacaacb (table test-table [id=178e673f35014379b6afd62b1124e29b]) successfully deleted
W20260808 00:32:57.625102 10952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:46351 (error 108)
I20260808 00:32:57.625393 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 11275
I20260808 00:32:57.629997 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:32:57.625577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:46351 (local address 127.0.0.1:52704)
0808 00:32:57.629861 (+ 4284us) negotiation.cc:107] Waiting for socket to connect
0808 00:32:57.629940 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4258,"thread_start_us":42,"threads_started":1}
W20260808 00:32:57.630149 10952 catalog_manager.cc:4584] TS ad79792420c548e8bc9da2e9cd2fa10b (127.31.210.193:46351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee37a80889ef431d9f8122a92dacaacb: Network error: Client connection negotiation failed: client connection to 127.31.210.193:46351: connect: Connection refused (error 111)
I20260808 00:32:57.636353 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 11456
I20260808 00:32:57.645215 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 10926
2026-08-08T00:32:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2943 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[11797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[11797](info): set up 2 sockets
Aug 08 00:32:57 dist-test-slave-44d2 krb5kdc[11797](info): commencing operation
krb5kdc: starting...
W20260808 00:32:58.943346 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:32:59 dist-test-slave-44d2 krb5kdc[11797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:32:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:32:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:32:59.081221 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44267
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:44267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:32:59.171376 11815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:32:59.171665 11815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:32:59.171706 11815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:32:59.172945 11815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:32:59.173018 11815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:32:59.173044 11815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:32:59.173074 11815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:32:59.173106 11815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:32:59.174742 11815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:44267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:32:59.175065 11815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:32:59.175316 11815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:32:59.180275 11822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:59.180339 11824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:32:59.180339 11820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:32:59.180583 11815 server_base.cc:1034] running on GCE node
I20260808 00:32:59.180820 11815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:32:59.181120 11815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:32:59.182318 11815 hybrid_clock.cc:648] HybridClock initialized: now 1786149179182305 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T00:32:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1540 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:32:59 dist-test-slave-44d2 krb5kdc[11834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:32:59 dist-test-slave-44d2 krb5kdc[11834](info): set up 2 sockets
Aug 08 00:32:59 dist-test-slave-44d2 krb5kdc[11834](info): commencing operation
krb5kdc: starting...
W20260808 00:33:00.466763 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:33:00 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:33:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:33:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:33:00.603914 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38865
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:00.696158 11851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:00.696424 11851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:00.696449 11851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:00.697744 11851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:33:00.697810 11851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:00.697825 11851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:00.697839 11851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:33:00.697850 11851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:33:00.699381 11851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:38865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:00.699663 11851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:00.699893 11851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:00.705560 11859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:00.705550 11856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:00.705588 11851 server_base.cc:1034] running on GCE node
W20260808 00:33:00.705528 11857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:00.705984 11851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:00.706301 11851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:00.707490 11851 hybrid_clock.cc:648] HybridClock initialized: now 1786149180707456 us; error 42 us; skew 500 ppm
Aug 08 00:33:00 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149180, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:00.710388 11851 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:33:00.710915 11851 webserver.cc:460] Webserver started at http://127.31.210.254:45505/ using document root <none> and password file <none>
I20260808 00:33:00.711131 11851 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:00.711177 11851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:00.711272 11851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:00.712380 11851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "74ea6293bdc141758deb1886438c3e5d"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "c2284c2b82a707c105acfbaea5554b2e"
server_key_iv: "dafffddafcf41234441bbf508ea26d61"
server_key_version: "encryptionkey@0"
I20260808 00:33:00.712862 11851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "74ea6293bdc141758deb1886438c3e5d"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "c2284c2b82a707c105acfbaea5554b2e"
server_key_iv: "dafffddafcf41234441bbf508ea26d61"
server_key_version: "encryptionkey@0"
I20260808 00:33:00.714254 11851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:00.715008 11868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:00.715209 11851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:00.715332 11851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "74ea6293bdc141758deb1886438c3e5d"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "c2284c2b82a707c105acfbaea5554b2e"
server_key_iv: "dafffddafcf41234441bbf508ea26d61"
server_key_version: "encryptionkey@0"
I20260808 00:33:00.715410 11851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:00.731148 11851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:00.733577 11851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:00.733762 11851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:00.738598 11851 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:38865
I20260808 00:33:00.738656 11953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:38865 every 8 connection(s)
I20260808 00:33:00.739151 11851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:33:00.740202 11957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:33:00.740620 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 11851
I20260808 00:33:00.740765 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:33:00.741016 32587 external_mini_cluster.cc:1428] Setting key e8026601a88d2deb2f86d1848f7f6104
I20260808 00:33:00.745643 11957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Bootstrap starting.
I20260808 00:33:00.746523 11957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Neither blocks nor log segments found. Creating new log.
I20260808 00:33:00.746882 11957 log.cc:824] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:00.747838 11957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: No bootstrap required, opened a new log
Aug 08 00:33:00 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:00.750914 11957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:00.751174 11957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:33:00.751230 11957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ea6293bdc141758deb1886438c3e5d, State: Initialized, Role: FOLLOWER
I20260808 00:33:00.751380 11957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:00.751492 11957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:33:00.751545 11957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:33:00.751597 11957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:33:00.752573 11957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:00.752719 11957 leader_election.cc:302] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74ea6293bdc141758deb1886438c3e5d; no voters:
I20260808 00:33:00.752928 11957 leader_election.cc:288] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:33:00.752962 11964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:33:00.753089 11964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 LEADER]: Becoming Leader. State: Replica: 74ea6293bdc141758deb1886438c3e5d, State: Running, Role: LEADER
I20260808 00:33:00.753181 11964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:00.753280 11957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:33:00.753604 11965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ea6293bdc141758deb1886438c3e5d. Latest consensus state: current_term: 1 leader_uuid: "74ea6293bdc141758deb1886438c3e5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } } }
I20260808 00:33:00.753679 11965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:00.753780 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:00.741627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.31.210.254:38865)
0808 00:33:00.741856 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:00.741867 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:00.741905 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:33:00.742629 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:00.742642 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:00.742742 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:00.742935 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:00.743708 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:00.744948 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:00.745759 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:00.746344 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:00.746488 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:00.749441 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:00.749446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:00.749446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:00.749468 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:00.751533 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:00.752047 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:00.752052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:00.752052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:00.752177 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:00.752410 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:00.752411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:00.752412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:00.752842 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:00.753028 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:00.753381 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 00:33:00.753541 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":100,"threads_started":1}
I20260808 00:33:00.753950 11969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:33:00.753566 11966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74ea6293bdc141758deb1886438c3e5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } } }
I20260808 00:33:00.754395 11966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:00.754665 11969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:33:00.757264 11969 catalog_manager.cc:1347] Generated new cluster ID: 43c95e729c2b4a269e69114778c62723
I20260808 00:33:00.757366 11969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:33:00.757562 11988 mvcc.cc:205] Tried to move back new op lower bound from 7316067044375232512 to 7316067044365344768. Current Snapshot: MvccSnapshot[applied={T|T < 7316067044375232512 or (T in {7316067044375232512})}]
I20260808 00:33:00.790194 11969 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:33:00.790942 11969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:33:00.812644 11969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Generated new TSK 0
I20260808 00:33:00.812940 11969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:33:00.852269 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38865
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:00.940373 11994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:00.940675 11994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:00.940716 11994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:00.941903 11994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:00.941984 11994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:00.942032 11994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:33:00.943544 11994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:00.943835 11994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:00.944087 11994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:00.949348 12004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:00.949323 12000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:00.949386 12001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:00.949697 11994 server_base.cc:1034] running on GCE node
I20260808 00:33:00.949952 11994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:00.950208 11994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:00.951397 11994 hybrid_clock.cc:648] HybridClock initialized: now 1786149180951371 us; error 45 us; skew 500 ppm
Aug 08 00:33:00 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149180, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:00.954371 11994 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:33:00.954876 11994 webserver.cc:460] Webserver started at http://127.31.210.193:40625/ using document root <none> and password file <none>
I20260808 00:33:00.955178 11994 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:00.955268 11994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:00.955406 11994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:00.956537 11994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "59e8836df8fd4a4d8b03fd7a49699855"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "4471ec93e484c5ab429734e6e7f1ec81"
server_key_iv: "4069b1631d6a43564ce22d96b1eb70b8"
server_key_version: "encryptionkey@0"
I20260808 00:33:00.956934 11994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "59e8836df8fd4a4d8b03fd7a49699855"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "4471ec93e484c5ab429734e6e7f1ec81"
server_key_iv: "4069b1631d6a43564ce22d96b1eb70b8"
server_key_version: "encryptionkey@0"
I20260808 00:33:00.958354 11994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:33:00.959297 12011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:00.959555 11994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:00.959672 11994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "59e8836df8fd4a4d8b03fd7a49699855"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "4471ec93e484c5ab429734e6e7f1ec81"
server_key_iv: "4069b1631d6a43564ce22d96b1eb70b8"
server_key_version: "encryptionkey@0"
I20260808 00:33:00.959746 11994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:00.972384 11994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:00.974828 11994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:00.975023 11994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:00.975265 11994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:00.975595 11994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:00.975651 11994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:00.975694 11994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:00.975721 11994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:00.981772 11994 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:37011
I20260808 00:33:00.981882 12177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:37011 every 8 connection(s)
I20260808 00:33:00.982191 11994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:33:00 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149180, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:00.989279 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 11994
I20260808 00:33:00.989401 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:33:00.989653 32587 external_mini_cluster.cc:1428] Setting key 6e5bc6b9ceaeef8168bd1ecccddbc6ab
I20260808 00:33:00.993008 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:00.983064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:50161 (local address 127.31.210.254:38865)
0808 00:33:00.983148 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:00.983150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:00.983347 (+ 197us) server_negotiation.cc:407] Connection header received
0808 00:33:00.984285 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:00.984286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:00.984295 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:00.984352 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:00.985107 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:00.985683 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:00.986753 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:00.986963 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:00.987074 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:00.989562 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:00.989564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:00.989564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:00.989574 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:00.991489 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:00.991917 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:00.991919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:00.991919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:00.991996 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:00.992234 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:00.992235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:00.992236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:00.992387 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:00.992465 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:00.992861 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 00:33:00.992936 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:33:00.993343 12180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:00.983024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:50161)
0808 00:33:00.983210 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:00.983255 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:00.984196 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:00.984414 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:00.984418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:00.984754 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:00.984952 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:00.984957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:00.985807 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:00.985808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:00.986655 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:00.986658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:00.987092 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:00.987093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:00.987225 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:00.987832 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:00.987841 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:00.989367 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:00.991578 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:00.991610 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:00.991612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:00.991839 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:00.992065 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:00.992065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:00.992065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:00.992170 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:00.992475 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:00.992515 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:00.992706 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 00:33:00.992837 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:33:00.993027 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260808 00:33:00.993948 12178 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38865
I20260808 00:33:00.994079 12178 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:00.994270 12178 heartbeater.cc:511] Master 127.31.210.254:38865 requested a full tablet report, sending...
I20260808 00:33:00.994781 11895 ts_manager.cc:194] Registered new tserver with Master: 59e8836df8fd4a4d8b03fd7a49699855 (127.31.210.193:37011)
I20260808 00:33:00.995441 11895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:50161
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:33:01.019862 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38865
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:01.105544 12184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:01.105779 12184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:01.105819 12184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:01.106993 12184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:01.107069 12184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:01.107116 12184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:33:01.108636 12184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:01.108907 12184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:01.109151 12184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:01.114627 12194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.114634 12191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:01.114773 12184 server_base.cc:1034] running on GCE node
W20260808 00:33:01.114653 12190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:01.115038 12184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:01.115293 12184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:01.116451 12184 hybrid_clock.cc:648] HybridClock initialized: now 1786149181116426 us; error 45 us; skew 500 ppm
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149181, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:01.119403 12184 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:33:01.119907 12184 webserver.cc:460] Webserver started at http://127.31.210.194:44043/ using document root <none> and password file <none>
I20260808 00:33:01.120102 12184 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:01.120143 12184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:01.120234 12184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:01.121337 12184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "5510c3877b944b03bb57e725db15997d"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "18a41f43dc57f6f42e9516d332dffb4f"
server_key_iv: "702bc2d179226cf42947103b0edcab2b"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.121660 12184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "5510c3877b944b03bb57e725db15997d"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "18a41f43dc57f6f42e9516d332dffb4f"
server_key_iv: "702bc2d179226cf42947103b0edcab2b"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.122946 12184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:33:01.123780 12202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.124064 12184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:01.124151 12184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "5510c3877b944b03bb57e725db15997d"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "18a41f43dc57f6f42e9516d332dffb4f"
server_key_iv: "702bc2d179226cf42947103b0edcab2b"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.124207 12184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:01.140749 12184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:01.142946 12184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:01.143131 12184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:01.143348 12184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:01.143621 12184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:01.143651 12184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.143674 12184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:01.143687 12184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.149808 12184 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:41247
I20260808 00:33:01.149890 12360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:41247 every 8 connection(s)
I20260808 00:33:01.150233 12184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:33:01.156075 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 12184
I20260808 00:33:01.156206 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:33:01.156468 32587 external_mini_cluster.cc:1428] Setting key 328e3569f67ddcde04bf3cf918f5d165
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149181, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:01.161473 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.151073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:46505 (local address 127.31.210.254:38865)
0808 00:33:01.151156 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.151159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.151273 (+ 114us) server_negotiation.cc:407] Connection header received
0808 00:33:01.151762 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.151763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.151774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.151824 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.152877 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.153396 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.154752 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.154945 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.155009 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.158041 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:01.158042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:01.158043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:01.158051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:01.159614 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:01.160165 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:01.160166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:01.160167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:01.160232 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:01.160580 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:01.160581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:01.160581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:01.160710 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:01.160775 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:01.161143 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 00:33:01.161276 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:33:01.161505 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.150949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:46505)
0808 00:33:01.151130 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.151182 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.151675 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.151888 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.151891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.152547 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.152785 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.152791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.153637 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.153650 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.154659 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.154662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.155235 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.155236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.155354 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.155996 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.156009 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.157880 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:01.159712 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:01.159720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:01.159722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:01.160062 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:01.160316 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:01.160317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:01.160318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:01.160471 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:01.160784 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:01.160791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:01.160969 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 00:33:01.161092 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 00:33:01.161279 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
I20260808 00:33:01.162220 12361 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38865
I20260808 00:33:01.162343 12361 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:01.162578 12361 heartbeater.cc:511] Master 127.31.210.254:38865 requested a full tablet report, sending...
I20260808 00:33:01.163048 11895 ts_manager.cc:194] Registered new tserver with Master: 5510c3877b944b03bb57e725db15997d (127.31.210.194:41247)
I20260808 00:33:01.163515 11895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:46505
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:33:01.187918 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38865
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:01.273605 12368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:01.273823 12368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:01.273847 12368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:01.274971 12368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:01.275008 12368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:01.275035 12368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:33:01.276432 12368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:01.276659 12368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:01.276852 12368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:01.282663 12375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.282727 12374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.282948 12377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:01.283063 12368 server_base.cc:1034] running on GCE node
I20260808 00:33:01.283316 12368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:01.283579 12368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:01.284722 12368 hybrid_clock.cc:648] HybridClock initialized: now 1786149181284706 us; error 36 us; skew 500 ppm
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149181, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:01.291340 12368 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:33:01.291973 12368 webserver.cc:460] Webserver started at http://127.31.210.195:35797/ using document root <none> and password file <none>
I20260808 00:33:01.292251 12368 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:01.292325 12368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:01.292522 12368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:01.293700 12368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "1f9c2a0e17fe4095b46170f68f64b6b5"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "c55280d1a8c10fe2f1abf7b87abf2ca5"
server_key_iv: "4314b20c466b7bbdaa83c2794907ef47"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.294080 12368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "1f9c2a0e17fe4095b46170f68f64b6b5"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "c55280d1a8c10fe2f1abf7b87abf2ca5"
server_key_iv: "4314b20c466b7bbdaa83c2794907ef47"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.295460 12368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:01.296218 12386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.296530 12368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:01.296644 12368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "1f9c2a0e17fe4095b46170f68f64b6b5"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "c55280d1a8c10fe2f1abf7b87abf2ca5"
server_key_iv: "4314b20c466b7bbdaa83c2794907ef47"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.296728 12368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:01.337939 12368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:01.340014 12368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:01.340199 12368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:01.340459 12368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:01.340869 12368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:01.340929 12368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.340971 12368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:01.341019 12368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.347303 12368 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:39585
I20260808 00:33:01.347358 12543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:39585 every 8 connection(s)
I20260808 00:33:01.347688 12368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:33:01.354377 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 12368
I20260808 00:33:01.354537 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:33:01.354754 32587 external_mini_cluster.cc:1428] Setting key ef78aafb82eb25c8db81dd925095068f
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149181, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:01.358348 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.348406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:43901 (local address 127.31.210.254:38865)
0808 00:33:01.348507 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.348509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.348763 (+ 254us) server_negotiation.cc:407] Connection header received
0808 00:33:01.349254 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.349255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.349266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.349319 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.350268 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.350766 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.351920 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.352112 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.352165 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.355443 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:01.355445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:01.355445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:01.355454 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:01.357110 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:01.357525 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:01.357526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:01.357526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:01.357579 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:01.357806 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:01.357807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:01.357807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:01.357910 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:01.357961 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:01.358214 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 00:33:01.358284 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:33:01.358521 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.348380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:43901)
0808 00:33:01.348601 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.348660 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.349166 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.349374 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.349377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.349991 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.350192 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.350196 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.350884 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.350886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.351829 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.351832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.352198 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.352199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.352335 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.353377 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.353394 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.355318 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:01.357205 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:01.357213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:01.357214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:01.357450 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:01.357641 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:01.357641 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:01.357642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:01.357749 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:01.357965 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:01.357970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:01.358094 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 00:33:01.358178 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:33:01.358307 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":127,"threads_started":1}
I20260808 00:33:01.359030 12544 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38865
I20260808 00:33:01.359148 12544 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:01.359331 12544 heartbeater.cc:511] Master 127.31.210.254:38865 requested a full tablet report, sending...
I20260808 00:33:01.359717 11895 ts_manager.cc:194] Registered new tserver with Master: 1f9c2a0e17fe4095b46170f68f64b6b5 (127.31.210.195:39585)
I20260808 00:33:01.360103 11895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:43901
I20260808 00:33:01.366721 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab.
Entry for principal oryx/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab.
WARNING: no policy specified for oryx/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab.
Entry for principal oryx/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab.
WARNING: no policy specified for oryx/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab.
Entry for principal oryx/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab.
I20260808 00:33:01.452684 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 11994
I20260808 00:33:01.467916 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 12184
I20260808 00:33:01.481034 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 12368
I20260808 00:33:01.502605 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 11851
I20260808 00:33:01.511322 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38865
--webserver_interface=127.31.210.254
--webserver_port=45505
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:01.596010 12553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:01.596227 12553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:01.596251 12553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:01.597467 12553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:33:01.597522 12553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:01.597537 12553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:01.597550 12553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:33:01.597563 12553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:33:01.599174 12553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:38865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=45505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:01.599450 12553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:01.599696 12553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:01.605120 12569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.605145 12567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:01.605310 12553 server_base.cc:1034] running on GCE node
W20260808 00:33:01.605144 12564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:01.605685 12553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:01.606058 12553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:01.607273 12553 hybrid_clock.cc:648] HybridClock initialized: now 1786149181607245 us; error 39 us; skew 500 ppm
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149181, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:01.610234 12553 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:33:01.610728 12553 webserver.cc:460] Webserver started at http://127.31.210.254:45505/ using document root <none> and password file <none>
I20260808 00:33:01.610994 12553 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:01.611088 12553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:01.612774 12553 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:33:01.613551 12577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.613750 12553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:01.613864 12553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "74ea6293bdc141758deb1886438c3e5d"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "c2284c2b82a707c105acfbaea5554b2e"
server_key_iv: "dafffddafcf41234441bbf508ea26d61"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.614243 12553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:01.624049 12553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:01.626082 12553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:01.626247 12553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:01.630239 12553 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:38865
I20260808 00:33:01.630300 12651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:38865 every 8 connection(s)
I20260808 00:33:01.630712 12553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:33:01.632274 12652 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:33:01.633052 12652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Bootstrap starting.
I20260808 00:33:01.637017 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 12553
I20260808 00:33:01.637332 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:37011
--local_ip_for_outbound_sockets=127.31.210.193
--tserver_master_addrs=127.31.210.254:38865
--webserver_port=40625
--webserver_interface=127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:33:01.642216 12652 log.cc:824] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:01.643908 12652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:33:01.644225 12652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Bootstrap complete.
I20260808 00:33:01.646667 12652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:01.646885 12652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ea6293bdc141758deb1886438c3e5d, State: Initialized, Role: FOLLOWER
I20260808 00:33:01.647060 12652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:01.647135 12652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:33:01.647197 12652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:33:01.647241 12652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:33:01.648252 12652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:01.648340 12652 leader_election.cc:302] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74ea6293bdc141758deb1886438c3e5d; no voters:
I20260808 00:33:01.648555 12652 leader_election.cc:288] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:33:01.648591 12658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:33:01.648723 12658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [term 2 LEADER]: Becoming Leader. State: Replica: 74ea6293bdc141758deb1886438c3e5d, State: Running, Role: LEADER
I20260808 00:33:01.648813 12658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } }
I20260808 00:33:01.648855 12652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:33:01.649073 12660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ea6293bdc141758deb1886438c3e5d. Latest consensus state: current_term: 2 leader_uuid: "74ea6293bdc141758deb1886438c3e5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } } }
I20260808 00:33:01.649389 12660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:01.649487 12659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "74ea6293bdc141758deb1886438c3e5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ea6293bdc141758deb1886438c3e5d" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38865 } } }
I20260808 00:33:01.649679 12659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:01.649777 12669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:33:01.650252 12669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:33:01.650380 12669 catalog_manager.cc:1259] Loaded cluster ID: 43c95e729c2b4a269e69114778c62723
I20260808 00:33:01.650614 12669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:33:01.651811 12669 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:33:01.652073 12669 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74ea6293bdc141758deb1886438c3e5d: Loaded TSK: 0
I20260808 00:33:01.652300 12669 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:33:01.726163 12654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:01.726393 12654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:01.726459 12654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:01.727720 12654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:01.727767 12654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:01.727795 12654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:33:01.729475 12654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.193.keytab
--principal=oryx/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:37011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=40625
--tserver_master_addrs=127.31.210.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:01.729761 12654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:01.730036 12654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:01.735546 12687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.735638 12684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.735728 12685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:01.735863 12654 server_base.cc:1034] running on GCE node
I20260808 00:33:01.736183 12654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:01.736515 12654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:01.737691 12654 hybrid_clock.cc:648] HybridClock initialized: now 1786149181737673 us; error 35 us; skew 500 ppm
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149181, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:01.740979 12654 init.cc:377] Logged in from keytab as oryx/127.31.210.193@KRBTEST.COM (short username oryx)
I20260808 00:33:01.741564 12654 webserver.cc:460] Webserver started at http://127.31.210.193:40625/ using document root <none> and password file <none>
I20260808 00:33:01.741861 12654 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:01.741968 12654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:01.743880 12654 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:33:01.744666 12698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.744850 12654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:01.744962 12654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "59e8836df8fd4a4d8b03fd7a49699855"
format_stamp: "Formatted at 2026-08-08 00:33:00 on dist-test-slave-44d2"
server_key: "4471ec93e484c5ab429734e6e7f1ec81"
server_key_iv: "4069b1631d6a43564ce22d96b1eb70b8"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.745364 12654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:01.758255 12654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:01.760937 12654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:01.761186 12654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:01.761466 12654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:01.761839 12654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:01.761876 12654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.761902 12654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:01.761950 12654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.768711 12654 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:37011
I20260808 00:33:01.768762 12860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:37011 every 8 connection(s)
I20260808 00:33:01.769141 12654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:33:01.774075 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 12654
I20260808 00:33:01.774442 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:41247
--local_ip_for_outbound_sockets=127.31.210.194
--tserver_master_addrs=127.31.210.254:38865
--webserver_port=44043
--webserver_interface=127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:01.780185 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.769985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:60721)
0808 00:33:01.770177 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.770223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.770798 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.771290 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.771294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.771961 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.772172 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.772177 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.774007 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.774009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.775514 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.775517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.776423 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.776425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.776636 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.777289 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.777301 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.780024 (+ 2723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
W20260808 00:33:01.780510 12863 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:38865 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.210.254:38865: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
W20260808 00:33:01.780714 12868 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:01.770058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:60721 (local address 127.31.210.254:38865)
0808 00:33:01.770283 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.770291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.770326 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:33:01.771029 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.771037 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.771094 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.771243 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.772312 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.773862 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.775629 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.776198 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.776311 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.780474 (+ 4163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:60721: BlockingRecv error: recv got EOF from 127.31.210.193:60721 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":108,"threads_started":1}
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:01.786744 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.780766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:48011)
0808 00:33:01.780817 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.780829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.780869 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.781142 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.781143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.781324 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.781390 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.781393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.782281 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.782283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.783381 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.783385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.783828 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.783830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.783917 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.784423 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.784430 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.786656 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:33:01.786806 12868 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:01.780873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:48011 (local address 127.31.210.254:38865)
0808 00:33:01.781007 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.781008 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.781017 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:33:01.781040 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.781041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.781052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.781114 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.781482 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.782182 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.783489 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.783737 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.783806 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.786739 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:48011: BlockingRecv error: recv got EOF from 127.31.210.193:48011 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":148,"threads_started":1}
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:01.791510 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.787067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:42259)
0808 00:33:01.787182 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.787194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.787242 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.787392 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.787393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.787526 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.787553 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.787554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.788171 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.788172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.788769 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.788771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.789120 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.789121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.789174 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.789614 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.789620 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.791421 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260808 00:33:01.791615 12868 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:01.787137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:42259 (local address 127.31.210.254:38865)
0808 00:33:01.787187 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.787189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.787227 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:33:01.787306 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.787307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.787317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.787387 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.787630 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.788090 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.788851 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.789039 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.789109 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.791534 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:42259: BlockingRecv error: recv got EOF from 127.31.210.193:42259 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 00:33:01.791687 12863 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:33:01.866464 12869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:01.866679 12869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:01.866704 12869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:01.867911 12869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:01.867960 12869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:01.868023 12869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:33:01.869508 12869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.194.keytab
--principal=oryx/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:41247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=44043
--tserver_master_addrs=127.31.210.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:01.869777 12869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:01.870024 12869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:01.875321 12880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.875334 12878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:01.875329 12877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:01.875576 12869 server_base.cc:1034] running on GCE node
I20260808 00:33:01.875825 12869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:01.876087 12869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:01.877233 12869 hybrid_clock.cc:648] HybridClock initialized: now 1786149181877220 us; error 30 us; skew 500 ppm
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149181, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:01.879907 12869 init.cc:377] Logged in from keytab as oryx/127.31.210.194@KRBTEST.COM (short username oryx)
I20260808 00:33:01.880445 12869 webserver.cc:460] Webserver started at http://127.31.210.194:44043/ using document root <none> and password file <none>
I20260808 00:33:01.880764 12869 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:01.880848 12869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:01.882393 12869 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:33:01.883073 12888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.883292 12869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:33:01.883404 12869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "5510c3877b944b03bb57e725db15997d"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "18a41f43dc57f6f42e9516d332dffb4f"
server_key_iv: "702bc2d179226cf42947103b0edcab2b"
server_key_version: "encryptionkey@0"
I20260808 00:33:01.883755 12869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:01.896818 12869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:01.899138 12869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:01.899348 12869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:01.899626 12869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:01.900044 12869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:01.900125 12869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.900173 12869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:01.900200 12869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:01.906899 12869 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:41247
I20260808 00:33:01.906958 13046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:41247 every 8 connection(s)
I20260808 00:33:01.907349 12869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:33:01.912236 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 12869
I20260808 00:33:01.912752 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:39585
--local_ip_for_outbound_sockets=127.31.210.195
--tserver_master_addrs=127.31.210.254:38865
--webserver_port=35797
--webserver_interface=127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:01.915726 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.908106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:43653)
0808 00:33:01.908293 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.908341 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.908913 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.909159 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.909162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.909792 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.910017 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.910023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.910712 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.910714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.911513 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.911515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.911896 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.911898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.912182 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.912827 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.912839 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.915572 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
W20260808 00:33:01.915802 12868 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:01.908140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:43653 (local address 127.31.210.254:38865)
0808 00:33:01.908184 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.908186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.908443 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:33:01.909019 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.909020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.909036 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.909099 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.910109 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.910602 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.911596 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.911782 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.911839 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.915718 (+ 3879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:43653: BlockingRecv error: recv got EOF from 127.31.210.194:43653 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:33:01.916093 13047 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:38865 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.210.254:38865: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:01.922555 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.916420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:38187)
0808 00:33:01.916517 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.916530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.916577 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.916708 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.916708 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.916876 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.916939 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.916941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.917842 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.917843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.918814 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.918818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.919350 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.919351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.919491 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.920108 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.920115 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.922436 (+ 2321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 00:33:01.923596 12868 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:01.916515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:38187 (local address 127.31.210.254:38865)
0808 00:33:01.916572 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.916574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.916585 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:33:01.916638 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.916638 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.916648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.916694 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.917050 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.917752 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.918943 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.919245 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.919329 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.923519 (+ 4190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:38187: BlockingRecv error: recv got EOF from 127.31.210.194:38187 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:01 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:01.928503 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:01.923144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:42235)
0808 00:33:01.923208 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:01.923222 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:01.923272 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:01.923464 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:01.923466 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:01.923637 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:01.923681 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.923684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.924397 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.924398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.925040 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:01.925042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.925501 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.925503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:01.925574 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.926129 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:01.926134 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:01.928390 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 00:33:01.928580 12871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:01.923242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:42235 (local address 127.31.210.254:38865)
0808 00:33:01.923306 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:01.923308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:01.923317 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:33:01.923343 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:01.923344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:01.923356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:01.923460 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:01.923761 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.924305 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.925140 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:01.925401 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:01.925500 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:01.928475 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:42235: BlockingRecv error: recv got EOF from 127.31.210.194:42235 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 00:33:01.928646 13047 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:33:02.002516 13050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:02.002722 13050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:02.002745 13050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:02.003921 13050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:02.003964 13050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:02.004021 13050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:33:02.006040 13050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/krb5kdc/oryx_127.31.210.195.keytab
--principal=oryx/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:39585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=35797
--tserver_master_addrs=127.31.210.254:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:02.006305 13050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:02.006546 13050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:02.011837 13061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:02.011861 13058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:02.011955 13050 server_base.cc:1034] running on GCE node
W20260808 00:33:02.011837 13056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:02.012336 13050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:02.012641 13050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:02.013809 13050 hybrid_clock.cc:648] HybridClock initialized: now 1786149182013800 us; error 43 us; skew 500 ppm
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149182, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:02.016458 13050 init.cc:377] Logged in from keytab as oryx/127.31.210.195@KRBTEST.COM (short username oryx)
I20260808 00:33:02.017042 13050 webserver.cc:460] Webserver started at http://127.31.210.195:35797/ using document root <none> and password file <none>
I20260808 00:33:02.017297 13050 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:02.017380 13050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:02.018888 13050 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:02.019619 13070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:02.019824 13050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:02.019912 13050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "1f9c2a0e17fe4095b46170f68f64b6b5"
format_stamp: "Formatted at 2026-08-08 00:33:01 on dist-test-slave-44d2"
server_key: "c55280d1a8c10fe2f1abf7b87abf2ca5"
server_key_iv: "4314b20c466b7bbdaa83c2794907ef47"
server_key_version: "encryptionkey@0"
I20260808 00:33:02.020285 13050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:02.034308 13050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:02.036849 13050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:02.037024 13050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:02.037279 13050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:02.037622 13050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:02.037664 13050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:02.037686 13050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:02.037699 13050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:02.043803 13050 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:39585
I20260808 00:33:02.043903 13226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:39585 every 8 connection(s)
I20260808 00:33:02.044234 13050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:33:02.050467 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 13050
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:33:02.052311 12871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:02.045089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:37253 (local address 127.31.210.254:38865)
0808 00:33:02.045172 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:02.045174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:02.045361 (+ 187us) server_negotiation.cc:407] Connection header received
0808 00:33:02.045846 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:02.045847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:02.045860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:02.045916 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:02.046850 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.047371 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.048452 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.048810 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.048875 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.052264 (+ 3389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:37253: BlockingRecv error: recv got EOF from 127.31.210.195:37253 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:33:02.052381 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:02.045044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:37253)
0808 00:33:02.045237 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:02.045272 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:02.045749 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:02.045969 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:02.045973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:02.046563 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:02.046766 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.046770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.047511 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.047512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.048316 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.048319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.048893 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.048894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.049014 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.049533 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:02.049544 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:02.052182 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
W20260808 00:33:02.052719 13227 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:38865 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.210.254:38865: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:02.058075 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:02.053089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:39199)
0808 00:33:02.053126 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:02.053138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:02.053181 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:02.053326 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:02.053327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:02.053544 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:02.053593 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.053595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.054564 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.054565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.055125 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.055127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.055563 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.055565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.055646 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.056011 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:02.056015 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:02.057968 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 00:33:02.058164 12871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:02.053131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:39199 (local address 127.31.210.254:38865)
0808 00:33:02.053175 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:02.053177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:02.053187 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:33:02.053247 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:02.053248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:02.053260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:02.053321 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:02.053703 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.054210 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.055241 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.055466 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.055561 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.058095 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:39199: BlockingRecv error: recv got EOF from 127.31.210.195:39199 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:33:02.059001 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:02.051163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.31.210.254:38865)
0808 00:33:02.051204 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:02.051205 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:02.051216 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:33:02.051241 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:02.051242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:02.051258 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:02.051328 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:02.051982 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.052851 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.053725 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.053934 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.054007 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.055255 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:02.055260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:02.055261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:02.055272 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:02.057294 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:02.057687 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:02.057690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:02.057690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:02.057792 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:02.058016 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:02.058017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:02.058017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:02.058380 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:02.058539 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:02.058785 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 00:33:02.058900 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:02.063297 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:02.058468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:46107)
0808 00:33:02.058513 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:02.058522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:02.058557 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:02.058701 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:02.058702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:02.058842 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:02.058877 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.058880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.059766 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.059768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.060642 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.060644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.060996 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.060997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.061045 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.061437 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:02.061442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:02.063249 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:33:02.063473 12871 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:02.058514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:46107 (local address 127.31.210.254:38865)
0808 00:33:02.058554 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:02.058556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:02.058564 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:33:02.058609 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:02.058610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:02.058620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:02.058689 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:02.058967 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.059559 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.060705 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.060881 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.060937 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.063349 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:46107: BlockingRecv error: recv got EOF from 127.31.210.195:46107 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 00:33:02.063524 13227 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:02.797382 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:02.792147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:57449)
0808 00:33:02.792313 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:02.792324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:02.792379 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:02.792535 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:02.792536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:02.792720 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:02.792772 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.792773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.793542 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.793543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.794113 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.794116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.794521 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.794523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.794590 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.795149 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:02.795156 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:02.797307 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
W20260808 00:33:02.797544 13233 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:02.792243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:57449 (local address 127.31.210.254:38865)
0808 00:33:02.792372 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:02.792374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:02.792384 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:33:02.792424 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:02.792425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:02.792435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:02.792520 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:02.792856 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.793450 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.794209 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.794416 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.794482 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.797439 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:57449: BlockingRecv error: recv got EOF from 127.31.210.193:57449 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:02 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:02.934218 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:02.929097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:57159)
0808 00:33:02.929274 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:02.929286 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:02.929385 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:02.929546 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:02.929548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:02.929728 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:02.929778 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.929780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.930455 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.930457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.931016 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:02.931018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.931415 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.931416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:02.931485 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.931940 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:02.931947 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:02.934147 (+ 2200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
W20260808 00:33:02.934314 13233 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:02.929187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:57159 (local address 127.31.210.254:38865)
0808 00:33:02.929276 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:02.929279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:02.929363 (+ 84us) server_negotiation.cc:407] Connection header received
0808 00:33:02.929459 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:02.929460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:02.929473 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:02.929533 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:02.929858 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.930367 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.931105 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:02.931323 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:02.931414 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:02.934247 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:57159: BlockingRecv error: recv got EOF from 127.31.210.194:57159 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 00:33:03 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:03 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:03.069518 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:03.064097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:44949)
0808 00:33:03.064256 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:03.064277 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:03.064372 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:03.064563 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:03.064565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:03.064778 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:03.064831 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:03.064832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.065626 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.065628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:03.066275 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:03.066276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.066722 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.066723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:03.066796 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:03.067321 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:03.067328 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:03.069397 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
W20260808 00:33:03.069653 13233 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:03.064159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:44949 (local address 127.31.210.254:38865)
0808 00:33:03.064215 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:03.064218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:03.064355 (+ 137us) server_negotiation.cc:407] Connection header received
0808 00:33:03.064436 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:03.064437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:03.064454 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:03.064555 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:03.064926 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.065530 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.066376 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.066620 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.066720 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:03.069537 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:44949: BlockingRecv error: recv got EOF from 127.31.210.195:44949 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 00:33:03 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:03 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:03.803449 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:03.798039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:45347)
0808 00:33:03.798229 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:03.798241 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:03.798303 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:03.798460 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:03.798461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:03.798640 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:03.798693 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:03.798694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.799537 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.799539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:03.800108 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:03.800110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.800586 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.800588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:03.800637 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:03.801216 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:03.801221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:03.803390 (+ 2169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
W20260808 00:33:03.803695 13238 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:03.798170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:45347 (local address 127.31.210.254:38865)
0808 00:33:03.798315 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:03.798316 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:03.798325 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:33:03.798353 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:03.798354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:03.798365 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:03.798441 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:03.798802 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.799426 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.800198 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.800417 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.800505 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:03.803615 (+ 3110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:45347: BlockingRecv error: recv got EOF from 127.31.210.193:45347 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
Aug 08 00:33:03 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:03 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:03.939527 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:03.934781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:40621)
0808 00:33:03.934907 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:03.934922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:03.934985 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:03.935105 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:03.935106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:03.935267 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:03.935319 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:03.935321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.935954 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.935954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:03.936532 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:03.936535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.937024 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.937025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:03.937069 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:03.937504 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:03.937508 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:03.939467 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
W20260808 00:33:03.939674 13238 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:03.934824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:40621 (local address 127.31.210.254:38865)
0808 00:33:03.934894 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:03.934896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:03.934965 (+ 69us) server_negotiation.cc:407] Connection header received
0808 00:33:03.935040 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:03.935041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:03.935054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:03.935100 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:03.935376 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.935886 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.936655 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:03.936930 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:03.937002 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:03.939586 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:40621: BlockingRecv error: recv got EOF from 127.31.210.194:40621 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 00:33:04 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:04 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:04.075157 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:04.070019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:36369)
0808 00:33:04.070192 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:04.070206 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:04.070275 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:04.070459 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:04.070460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:04.070643 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:04.070695 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:04.070697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.071377 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.071379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:04.071937 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:04.071940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.072325 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.072326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:04.072412 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:04.073102 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:04.073107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:04.075036 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":65,"threads_started":1}
W20260808 00:33:04.075229 13238 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:04.070245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:36369 (local address 127.31.210.254:38865)
0808 00:33:04.070315 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:04.070318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:04.070327 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:33:04.070361 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:04.070362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:04.070374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:04.070430 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:04.070768 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.071285 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.072037 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.072224 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.072279 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:04.075152 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:36369: BlockingRecv error: recv got EOF from 127.31.210.195:36369 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 00:33:04 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:04 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:04.808930 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:04.803998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:59909)
0808 00:33:04.804190 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:04.804203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:04.804264 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:04.804437 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:04.804438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:04.804748 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:04.804816 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:04.804819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.805531 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.805532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:04.806057 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:04.806059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.806401 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.806402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:04.806449 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:04.807000 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:04.807006 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:04.808868 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":112,"threads_started":1}
W20260808 00:33:04.809072 13243 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:04.804137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:59909 (local address 127.31.210.254:38865)
0808 00:33:04.804247 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:04.804249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:04.804258 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:33:04.804316 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:04.804317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:04.804327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:04.804384 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:04.804904 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.805416 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.806138 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.806331 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.806386 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:04.808966 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:59909: BlockingRecv error: recv got EOF from 127.31.210.193:59909 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
Aug 08 00:33:04 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:04 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:04.945412 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:04.940118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:35191)
0808 00:33:04.940258 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:04.940269 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:04.940338 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:04.940533 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:04.940534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:04.940714 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:04.940794 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:04.940796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.941435 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.941436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:04.941974 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:04.941976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.942413 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.942414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:04.942524 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:04.943150 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:04.943155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:04.945336 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260808 00:33:04.945573 13243 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:04.940217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:35191 (local address 127.31.210.254:38865)
0808 00:33:04.940262 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:04.940264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:04.940323 (+ 59us) server_negotiation.cc:407] Connection header received
0808 00:33:04.940397 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:04.940398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:04.940408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:04.940473 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:04.940866 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.941366 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.942057 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:04.942257 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:04.942319 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:04.945481 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:35191: BlockingRecv error: recv got EOF from 127.31.210.194:35191 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 00:33:05 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:05 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:05.080788 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:05.075719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:34007)
0808 00:33:05.075875 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:05.075886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:05.075943 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:05.076082 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:05.076084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:05.076252 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:05.076305 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:05.076307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.077088 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.077090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:05.077642 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:05.077645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.078029 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.078031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:05.078086 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:05.078669 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:05.078674 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:05.080724 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260808 00:33:05.081012 13243 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:05.075821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:34007 (local address 127.31.210.254:38865)
0808 00:33:05.075871 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:05.075875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:05.075937 (+ 62us) server_negotiation.cc:407] Connection header received
0808 00:33:05.076003 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:05.076004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:05.076015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:05.076056 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:05.076399 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.076931 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.077730 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.077914 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.078016 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:05.080859 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:34007: BlockingRecv error: recv got EOF from 127.31.210.195:34007 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 00:33:05 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:05 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:05.814571 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:05.809423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:56089)
0808 00:33:05.809690 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:05.809702 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:05.809768 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:05.809924 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:05.809925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:05.810114 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:05.810169 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:05.810171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.810812 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.810813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:05.811390 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:05.811392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.811758 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.811759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:05.811801 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:05.812335 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:05.812341 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:05.814441 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":157,"threads_started":1}
W20260808 00:33:05.814592 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:05.809562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:56089 (local address 127.31.210.254:38865)
0808 00:33:05.809697 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:05.809699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:05.809780 (+ 81us) server_negotiation.cc:407] Connection header received
0808 00:33:05.809823 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:05.809824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:05.809836 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:05.809886 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:05.810239 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.810742 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.811473 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.811685 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.811746 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:05.814528 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:56089: BlockingRecv error: recv got EOF from 127.31.210.193:56089 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
Aug 08 00:33:05 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:05 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:05.951781 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:05.946212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:50549)
0808 00:33:05.946360 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:05.946372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:05.946442 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:05.946620 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:05.946621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:05.946795 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:05.946859 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:05.946860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.947575 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.947577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:05.948198 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:05.948201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.948676 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.948677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:05.948724 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:05.949523 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:05.949528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:05.951718 (+ 2190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
W20260808 00:33:05.952036 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:05.946296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:50549 (local address 127.31.210.254:38865)
0808 00:33:05.946376 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:05.946378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:05.946450 (+ 72us) server_negotiation.cc:407] Connection header received
0808 00:33:05.946519 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:05.946520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:05.946538 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:05.946601 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:05.946940 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.947471 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.948301 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:05.948563 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:05.948680 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:05.951936 (+ 3256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:50549: BlockingRecv error: recv got EOF from 127.31.210.194:50549 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 00:33:06 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:06 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:06.087213 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:06.081361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:53253)
0808 00:33:06.081652 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:06.081664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:06.081764 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:06.081933 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:06.081934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:06.082192 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:06.082278 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:06.082281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.083197 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.083199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:06.083745 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:06.083747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.084424 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.084425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:06.084524 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:06.085014 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:06.085019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:06.087141 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":199,"threads_started":1}
W20260808 00:33:06.087358 13247 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:06.081532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:53253 (local address 127.31.210.254:38865)
0808 00:33:06.081633 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:06.081636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:06.081741 (+ 105us) server_negotiation.cc:407] Connection header received
0808 00:33:06.081834 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:06.081835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:06.081849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:06.081924 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:06.082357 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.083100 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.083842 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.084278 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.084399 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:06.087246 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:53253: BlockingRecv error: recv got EOF from 127.31.210.195:53253 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 00:33:06 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:06 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:06.826464 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:06.815124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:45013)
0808 00:33:06.815284 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:06.815295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:06.815344 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:06.815535 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:06.815536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:06.815704 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:06.815757 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:06.815758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.816803 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.816805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:06.817556 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:06.817559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.817988 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.817990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:06.818056 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:06.824151 (+ 6095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:06.824158 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:06.826399 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
W20260808 00:33:06.826639 13252 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:06.815252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:45013 (local address 127.31.210.254:38865)
0808 00:33:06.815394 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:06.815396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:06.815404 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:33:06.815430 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:06.815431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:06.815442 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:06.815511 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:06.815833 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.816684 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.817655 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.817857 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.817917 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:06.826549 (+ 8632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:45013: BlockingRecv error: recv got EOF from 127.31.210.193:45013 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
Aug 08 00:33:06 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:06 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:06.957540 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:06.952646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:53121)
0808 00:33:06.952809 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:06.952821 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:06.952901 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:06.953066 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:06.953067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:06.953253 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:06.953307 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:06.953309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.953936 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.953937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:06.954499 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:06.954501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.954983 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.954983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:06.955023 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:06.955488 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:06.955493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:06.957455 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
W20260808 00:33:06.957607 13252 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:06.952761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:53121 (local address 127.31.210.254:38865)
0808 00:33:06.952838 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:06.952841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:06.952880 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:33:06.952962 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:06.952963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:06.952982 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:06.953047 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:06.953374 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.953867 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.954601 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:06.954834 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:06.954986 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:06.957558 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:53121: BlockingRecv error: recv got EOF from 127.31.210.194:53121 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 00:33:07 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:07 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:07.093240 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:07.087702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:37961)
0808 00:33:07.087899 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:07.087913 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:07.088019 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:07.088201 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:07.088202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:07.088531 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:07.088602 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:07.088605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.089604 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.089606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:07.090174 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:07.090177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.090544 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.090545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:07.090588 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:07.091069 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:07.091074 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:07.093170 (+ 2096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
W20260808 00:33:07.093312 13252 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:07.087818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:37961 (local address 127.31.210.254:38865)
0808 00:33:07.087898 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:07.087900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:07.088041 (+ 141us) server_negotiation.cc:407] Connection header received
0808 00:33:07.088085 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:07.088086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:07.088103 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:07.088171 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:07.088701 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.089514 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.090254 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.090467 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.090524 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:07.093262 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:37961: BlockingRecv error: recv got EOF from 127.31.210.195:37961 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 00:33:07 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:07 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:07.831867 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:07.826927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:37205)
0808 00:33:07.827172 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:07.827184 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:07.827249 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:07.827381 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:07.827382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:07.827576 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:07.827628 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:07.827630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.828280 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.828281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:07.828883 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:07.828885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.829235 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.829235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:07.829280 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:07.829798 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:07.829803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:07.831804 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":184,"threads_started":1}
W20260808 00:33:07.832015 13255 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:07.827009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:37205 (local address 127.31.210.254:38865)
0808 00:33:07.827189 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:07.827193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:07.827235 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:33:07.827295 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:07.827296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:07.827306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:07.827366 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:07.827703 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.828213 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.828964 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.829164 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.829218 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:07.831902 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:37205: BlockingRecv error: recv got EOF from 127.31.210.193:37205 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
Aug 08 00:33:07 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:07 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:07.962735 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:07.958034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:39949)
0808 00:33:07.958185 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:07.958196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:07.958263 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:07.958377 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:07.958378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:07.958566 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:07.958619 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:07.958621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.959275 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.959276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:07.959843 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:07.959845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.960201 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.960202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:07.960246 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:07.960771 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:07.960775 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:07.962671 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
W20260808 00:33:07.962855 13255 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:07.958119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:39949 (local address 127.31.210.254:38865)
0808 00:33:07.958198 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:07.958205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:07.958258 (+ 53us) server_negotiation.cc:407] Connection header received
0808 00:33:07.958310 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:07.958311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:07.958322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:07.958370 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:07.958687 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.959194 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.959919 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:07.960132 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:07.960186 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:07.962772 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:39949: BlockingRecv error: recv got EOF from 127.31.210.194:39949 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 00:33:08 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:08 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:08.098963 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:08.093807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:59323)
0808 00:33:08.093992 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:08.094003 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:08.094063 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:08.094208 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:08.094209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:08.094395 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:08.094450 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:08.094452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.095282 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.095283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:08.095888 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:08.095890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.096257 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.096258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:08.096304 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:08.096936 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:08.096941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:08.098898 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
W20260808 00:33:08.099125 13255 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:08.093887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:59323 (local address 127.31.210.254:38865)
0808 00:33:08.093965 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:08.093968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:08.094047 (+ 79us) server_negotiation.cc:407] Connection header received
0808 00:33:08.094118 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:08.094119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:08.094130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:08.094183 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:08.094561 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.095206 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.095970 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.096185 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.096239 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:08.098995 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:59323: BlockingRecv error: recv got EOF from 127.31.210.195:59323 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 00:33:08 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:08 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:08.837808 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:08.832465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:37663)
0808 00:33:08.832682 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:08.832694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:08.832753 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:08.832907 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:08.832908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:08.833110 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:08.833164 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:08.833166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.833864 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.833865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:08.834451 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:08.834453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.834791 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.834792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:08.834840 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:08.835428 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:08.835435 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:08.837741 (+ 2306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
W20260808 00:33:08.838064 13267 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:08.832622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:37663 (local address 127.31.210.254:38865)
0808 00:33:08.832765 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:08.832767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:08.832775 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:33:08.832805 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:08.832805 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:08.832818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:08.832882 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:08.833270 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.833787 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.834529 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.834722 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.834772 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:08.837936 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:37663: BlockingRecv error: recv got EOF from 127.31.210.193:37663 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
Aug 08 00:33:08 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:08 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:08.968511 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:08.963421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:39225)
0808 00:33:08.963658 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:08.963669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:08.963731 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:08.963866 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:08.963867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:08.964044 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:08.964098 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:08.964099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.964812 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.964813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:08.965355 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:08.965357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.965683 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.965684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:08.965730 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:08.966266 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:08.966271 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:08.968392 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":164,"threads_started":1}
W20260808 00:33:08.968622 13267 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:08.963560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:39225 (local address 127.31.210.254:38865)
0808 00:33:08.963632 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:08.963634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:08.963726 (+ 92us) server_negotiation.cc:407] Connection header received
0808 00:33:08.963775 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:08.963776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:08.963790 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:08.963847 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:08.964160 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.964741 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.965430 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:08.965615 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:08.965666 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:08.968548 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:39225: BlockingRecv error: recv got EOF from 127.31.210.194:39225 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 00:33:09 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:09 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:09.104403 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:09.099515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:55429)
0808 00:33:09.099698 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:09.099713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:09.099801 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:09.100012 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:09.100014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:09.100198 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:09.100255 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:09.100258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.101021 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.101023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:09.101562 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:09.101564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.101950 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.101951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:09.102026 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:09.102498 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:09.102502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:09.104343 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
W20260808 00:33:09.104570 13267 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:09.099594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:55429 (local address 127.31.210.254:38865)
0808 00:33:09.099679 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:09.099682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:09.099781 (+ 99us) server_negotiation.cc:407] Connection header received
0808 00:33:09.099877 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:09.099878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:09.099893 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:09.099941 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:09.100346 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.100931 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.101662 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.101873 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.101950 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:09.104464 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:55429: BlockingRecv error: recv got EOF from 127.31.210.195:55429 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 00:33:09 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:09 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:09.843562 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:09.838325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:39625)
0808 00:33:09.838587 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:09.838597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:09.838648 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:09.838811 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:09.838812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:09.839001 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:09.839060 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:09.839061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.839786 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.839788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:09.840363 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:09.840365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.840804 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.840806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:09.840860 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:09.841361 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:09.841366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:09.843445 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":194,"threads_started":1}
W20260808 00:33:09.843623 13272 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:09.838511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:39625 (local address 127.31.210.254:38865)
0808 00:33:09.838671 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:09.838672 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:09.838681 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:33:09.838706 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:09.838707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:09.838719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:09.838796 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:09.839162 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.839687 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.840466 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.840708 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.840768 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:09.843538 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:39625: BlockingRecv error: recv got EOF from 127.31.210.193:39625 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
Aug 08 00:33:09 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:09 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:09.974328 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:09.969217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:45809)
0808 00:33:09.969402 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:09.969415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:09.969485 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:09.969719 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:09.969720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:09.969978 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:09.970046 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:09.970049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.970880 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.970881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:09.971414 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:09.971416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.971743 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.971744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:09.971787 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:09.972239 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:09.972244 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:09.974227 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":90,"threads_started":1}
W20260808 00:33:09.974375 13272 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:09.969296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:45809 (local address 127.31.210.254:38865)
0808 00:33:09.969357 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:09.969360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:09.969489 (+ 129us) server_negotiation.cc:407] Connection header received
0808 00:33:09.969617 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:09.969618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:09.969634 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:09.969694 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:09.970148 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.970803 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.971492 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:09.971678 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:09.971730 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:09.974311 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:45809: BlockingRecv error: recv got EOF from 127.31.210.194:45809 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 00:33:10 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:10 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:10.109794 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:10.104906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:33433)
0808 00:33:10.105117 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:10.105130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:10.105196 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:10.105348 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:10.105349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:10.105529 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:10.105581 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:10.105583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.106311 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.106312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:10.106848 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:10.106850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.107269 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.107270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:10.107313 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:10.107753 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:10.107758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:10.109715 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":154,"threads_started":1}
W20260808 00:33:10.109898 13272 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:10.105021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:33433 (local address 127.31.210.254:38865)
0808 00:33:10.105094 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:10.105096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:10.105199 (+ 103us) server_negotiation.cc:407] Connection header received
0808 00:33:10.105255 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:10.105255 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:10.105270 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:10.105313 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:10.105697 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.106225 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.106998 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.107191 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.107256 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:10.109824 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:33433: BlockingRecv error: recv got EOF from 127.31.210.195:33433 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 00:33:10 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:10 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:10.849622 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:10.844271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:38569)
0808 00:33:10.844541 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:10.844562 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:10.844625 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:10.844832 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:10.844833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:10.845067 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:10.845121 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:10.845123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.845802 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.845804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:10.846428 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:10.846430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.846787 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.846788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:10.846831 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:10.847363 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:10.847368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:10.849512 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":123,"threads_started":1}
W20260808 00:33:10.849712 13278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:10.844424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:38569 (local address 127.31.210.254:38865)
0808 00:33:10.844608 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:10.844610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:10.844620 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:33:10.844713 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:10.844713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:10.844725 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:10.844790 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:10.845206 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.845721 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.846516 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.846715 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.846772 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:10.849628 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:38569: BlockingRecv error: recv got EOF from 127.31.210.193:38569 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":124,"threads_started":1}
Aug 08 00:33:10 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:10 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:10.979984 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:10.974850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:33311)
0808 00:33:10.974996 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:10.975006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:10.975065 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:10.975193 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:10.975193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:10.975370 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:10.975423 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:10.975424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.976243 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.976245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:10.976870 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:10.976873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.977246 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.977248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:10.977293 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:10.977771 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:10.977775 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:10.979861 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
W20260808 00:33:10.980011 13278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:10.974926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:33311 (local address 127.31.210.254:38865)
0808 00:33:10.975014 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:10.975016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:10.975069 (+ 53us) server_negotiation.cc:407] Connection header received
0808 00:33:10.975112 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:10.975112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:10.975122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:10.975185 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:10.975522 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.976131 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.976973 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:10.977170 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:10.977238 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:10.979945 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:33311: BlockingRecv error: recv got EOF from 127.31.210.194:33311 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 00:33:11 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:11 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:11.115778 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:11.110457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:51193)
0808 00:33:11.110612 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:11.110624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:11.110700 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:11.110865 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:11.110867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:11.111103 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:11.111156 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:11.111159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.111931 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.111933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:11.112530 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:11.112533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.112951 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.112953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:11.113024 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:11.113524 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:11.113529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:11.115678 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260808 00:33:11.115823 13278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:11.110540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:51193 (local address 127.31.210.254:38865)
0808 00:33:11.110625 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:11.110627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:11.110734 (+ 107us) server_negotiation.cc:407] Connection header received
0808 00:33:11.110765 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:11.110766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:11.110781 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:11.110845 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:11.111268 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.111822 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.112653 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.112857 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.112947 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:11.115758 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:51193: BlockingRecv error: recv got EOF from 127.31.210.195:51193 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 00:33:11 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:11 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:11.860414 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:11.850175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:47871)
0808 00:33:11.850328 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:11.850339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:11.850392 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:11.850571 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:11.850572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:11.850787 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:11.850840 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:11.850842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.851512 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.851513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:11.852090 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:11.852092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.852450 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.852451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:11.852551 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:11.858112 (+ 5561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:11.858119 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:11.860351 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
W20260808 00:33:11.860668 13283 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:11.850267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:47871 (local address 127.31.210.254:38865)
0808 00:33:11.850410 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:11.850412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:11.850422 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:33:11.850451 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:11.850452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:11.850463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:11.850554 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:11.850916 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.851422 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.852176 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.852378 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.852434 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:11.860588 (+ 8154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:47871: BlockingRecv error: recv got EOF from 127.31.210.193:47871 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
Aug 08 00:33:11 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:11 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:11.986171 13284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:11.980626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:43063)
0808 00:33:11.980839 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:11.980851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:11.980999 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:11.981147 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:11.981148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:11.981338 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:11.981413 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:11.981415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.982144 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.982145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:11.982719 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:11.982721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.983167 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.983168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:11.983231 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:11.983694 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:11.983699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:11.986046 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":120,"threads_started":1}
W20260808 00:33:11.986280 13283 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:11.980716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:43063 (local address 127.31.210.254:38865)
0808 00:33:11.980811 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:11.980813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:11.980998 (+ 185us) server_negotiation.cc:407] Connection header received
0808 00:33:11.981054 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:11.981055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:11.981067 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:11.981120 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:11.981523 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.982037 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.982814 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:11.983063 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:11.983126 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:11.986200 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:43063: BlockingRecv error: recv got EOF from 127.31.210.194:43063 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 00:33:12 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:12 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:12.121821 13285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:12.116393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:53311)
0808 00:33:12.116559 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:12.116572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:12.116669 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:12.116802 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:12.116803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:12.117016 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:12.117066 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:12.117068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.117860 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.117862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:12.118584 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:12.118589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.119072 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.119074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:12.119170 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:12.119742 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:12.119748 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:12.121759 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
W20260808 00:33:12.121968 13283 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:12.116448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:53311 (local address 127.31.210.254:38865)
0808 00:33:12.116546 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:12.116549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:12.116650 (+ 101us) server_negotiation.cc:407] Connection header received
0808 00:33:12.116729 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:12.116730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:12.116744 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:12.116798 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:12.117179 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.117759 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.118665 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.118945 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.119023 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:12.121894 (+ 2871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:53311: BlockingRecv error: recv got EOF from 127.31.210.195:53311 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 00:33:12 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:12 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:12.866851 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:12.860995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:42757)
0808 00:33:12.861222 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:12.861236 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:12.861298 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:12.861661 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:12.861663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:12.861973 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:12.862059 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:12.862062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.863070 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.863072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:12.863681 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:12.863683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.864136 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.864137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:12.864193 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:12.864723 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:12.864728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:12.866733 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":109,"threads_started":1}
W20260808 00:33:12.866899 13287 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:12.861101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:42757 (local address 127.31.210.254:38865)
0808 00:33:12.861262 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:12.861264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:12.861278 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:33:12.861512 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:12.861514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:12.861531 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:12.861642 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:12.862152 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.863000 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.863800 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.864033 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.864107 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:12.866824 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:42757: BlockingRecv error: recv got EOF from 127.31.210.193:42757 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
Aug 08 00:33:12 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:12 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:12.991883 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:12.986732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:32791)
0808 00:33:12.986948 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:12.986960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:12.987055 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:12.987217 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:12.987218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:12.987423 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:12.987473 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:12.987474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.988229 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.988230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:12.988814 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:12.988816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.989236 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.989238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:12.989306 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:12.989732 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:12.989737 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:12.991770 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
W20260808 00:33:12.991917 13287 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:12.986826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:32791 (local address 127.31.210.254:38865)
0808 00:33:12.986920 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:12.986923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:12.987041 (+ 118us) server_negotiation.cc:407] Connection header received
0808 00:33:12.987119 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:12.987120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:12.987133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:12.987210 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:12.987561 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.988130 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.988908 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:12.989141 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:12.989231 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:12.991853 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:32791: BlockingRecv error: recv got EOF from 127.31.210.194:32791 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 00:33:13 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:13 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:13.127107 13290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:13.122357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:33335)
0808 00:33:13.122503 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:13.122521 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:13.122582 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:13.122808 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:13.122809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:13.122994 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:13.123046 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:13.123048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.123754 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.123755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:13.124377 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:13.124379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.124748 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.124749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:13.124794 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:13.125236 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:13.125241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:13.127013 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
W20260808 00:33:13.127166 13287 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:13.122418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:33335 (local address 127.31.210.254:38865)
0808 00:33:13.122490 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:13.122493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:13.122584 (+ 91us) server_negotiation.cc:407] Connection header received
0808 00:33:13.122670 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:13.122671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:13.122688 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:13.122737 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:13.123141 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.123657 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.124462 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.124679 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.124737 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:13.127117 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:33335: BlockingRecv error: recv got EOF from 127.31.210.195:33335 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 00:33:13 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:13 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:13.873025 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:13.867470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:45559)
0808 00:33:13.867627 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:13.867645 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:13.867705 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:13.867869 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:13.867870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:13.868096 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:13.868154 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:13.868156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.868879 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.868880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:13.869561 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:13.869565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.870047 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.870048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:13.870111 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:13.870674 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:13.870679 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:13.872891 (+ 2212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
W20260808 00:33:13.873056 13292 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:13.867547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:45559 (local address 127.31.210.254:38865)
0808 00:33:13.867691 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:13.867692 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:13.867702 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:33:13.867761 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:13.867762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:13.867773 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:13.867839 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:13.868245 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.868778 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.869682 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.869952 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.870027 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:13.872989 (+ 2962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:45559: BlockingRecv error: recv got EOF from 127.31.210.193:45559 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
Aug 08 00:33:13 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:13 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:13.997707 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:13.992488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:34879)
0808 00:33:13.992678 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:13.992691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:13.992771 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:13.992950 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:13.992952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:13.993253 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:13.993319 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:13.993322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.994070 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.994071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:13.994625 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:13.994627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.994972 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.994973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:13.995016 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:13.995540 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:13.995546 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:13.997631 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":125,"threads_started":1}
W20260808 00:33:13.997814 13292 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:13.992617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:34879 (local address 127.31.210.254:38865)
0808 00:33:13.992681 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:13.992683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:13.992783 (+ 100us) server_negotiation.cc:407] Connection header received
0808 00:33:13.992826 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:13.992827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:13.992840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:13.992917 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:13.993413 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.993933 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.994710 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:13.994901 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:13.994958 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:13.997741 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:34879: BlockingRecv error: recv got EOF from 127.31.210.194:34879 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 00:33:14 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:14 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:14.132634 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:14.127660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:42987)
0808 00:33:14.127832 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:14.127844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:14.127911 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:14.128042 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:14.128042 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:14.128220 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:14.128276 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:14.128277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.129039 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.129040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:14.129643 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:14.129645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.130008 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.130009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:14.130062 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:14.130577 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:14.130582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:14.132560 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
W20260808 00:33:14.132827 13292 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:14.127738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:42987 (local address 127.31.210.254:38865)
0808 00:33:14.127804 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:14.127807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:14.127913 (+ 106us) server_negotiation.cc:407] Connection header received
0808 00:33:14.127956 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:14.127957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:14.127971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:14.128035 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:14.128340 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.128912 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.129730 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.129916 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.129972 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:14.132692 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:42987: BlockingRecv error: recv got EOF from 127.31.210.195:42987 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 00:33:14 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:14 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:14.879137 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:14.873663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:60241)
0808 00:33:14.873939 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:14.874012 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:14.874090 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:14.874248 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:14.874249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:14.874439 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:14.874491 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:14.874492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.875368 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.875370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:14.875987 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:14.875990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.876396 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.876397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:14.876463 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:14.877108 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:14.877113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:14.879020 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":85,"threads_started":1}
W20260808 00:33:14.879207 13298 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:14.873880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:60241 (local address 127.31.210.254:38865)
0808 00:33:14.874060 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:14.874062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:14.874075 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:33:14.874144 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:14.874144 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:14.874157 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:14.874220 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:14.874597 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.875260 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.876102 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.876316 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.876395 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:14.879118 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:60241: BlockingRecv error: recv got EOF from 127.31.210.193:60241 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":74,"threads_started":1}
Aug 08 00:33:14 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:14 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:15.003110 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:14.998266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:34097)
0808 00:33:14.998442 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:14.998455 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:14.998528 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:14.998682 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:14.998683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:14.998864 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:14.998919 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:14.998920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.999595 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:14.999596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:15.000168 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:15.000170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.000622 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.000623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:15.000670 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:15.001179 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:15.001183 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:15.003041 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":63,"threads_started":1}
W20260808 00:33:15.003242 13298 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:14.998470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:34097 (local address 127.31.210.254:38865)
0808 00:33:14.998566 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:14.998568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:14.998576 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:33:14.998600 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:14.998601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:14.998611 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:14.998664 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:14.998986 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:14.999492 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.000279 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.000548 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.000620 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:15.003160 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:34097: BlockingRecv error: recv got EOF from 127.31.210.194:34097 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 00:33:15 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:15 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:15.138604 13301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:15.133265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:53297)
0808 00:33:15.133488 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:15.133502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:15.133577 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:15.133741 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:15.133742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:15.134054 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:15.134108 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:15.134110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.134846 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.134848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:15.135544 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:15.135548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.135909 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.135910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:15.135964 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:15.136532 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:15.136537 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:15.138490 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":123,"threads_started":1}
W20260808 00:33:15.138634 13298 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:15.133322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:53297 (local address 127.31.210.254:38865)
0808 00:33:15.133399 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:15.133402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:15.133622 (+ 220us) server_negotiation.cc:407] Connection header received
0808 00:33:15.133652 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:15.133653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:15.133667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:15.133711 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:15.134210 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.134743 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.135637 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.135821 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.135877 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:15.138585 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:53297: BlockingRecv error: recv got EOF from 127.31.210.195:53297 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 00:33:15 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:15 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:15.884946 13302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:15.879760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:41993)
0808 00:33:15.880124 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:15.880143 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:15.880227 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:15.880410 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:15.880411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:15.880628 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:15.880678 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:15.880680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.881367 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.881368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:15.881922 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:15.881924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.882328 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.882329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:15.882397 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:15.882896 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:15.882901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:15.884885 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":279,"threads_started":1}
W20260808 00:33:15.885113 13303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:15.879821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:41993 (local address 127.31.210.254:38865)
0808 00:33:15.880100 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:15.880103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:15.880210 (+ 107us) server_negotiation.cc:407] Connection header received
0808 00:33:15.880306 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:15.880307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:15.880322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:15.880403 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:15.880769 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.881280 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.882036 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:15.882244 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:15.882323 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:15.885021 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:41993: BlockingRecv error: recv got EOF from 127.31.210.193:41993 (error 108)
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":215,"threads_started":1}
Aug 08 00:33:16 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:16 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:16.009147 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:16.003763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:38609)
0808 00:33:16.003970 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:16.003983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:16.004061 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:16.004205 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:16.004206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:16.004390 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:16.004443 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:16.004450 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.005175 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.005177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:16.005737 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:16.005739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.006212 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.006214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:16.006284 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:16.006881 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:16.006887 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:16.009047 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
W20260808 00:33:16.009208 13303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:16.003844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:38609 (local address 127.31.210.254:38865)
0808 00:33:16.003921 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:16.003923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:16.004073 (+ 150us) server_negotiation.cc:407] Connection header received
0808 00:33:16.004120 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:16.004121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:16.004133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:16.004177 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:16.004598 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.005100 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.005846 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.006106 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.006184 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:16.009142 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:38609: BlockingRecv error: recv got EOF from 127.31.210.194:38609 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 00:33:16 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:16 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.195@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:16.144050 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:16.139159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.195:40137)
0808 00:33:16.139316 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:16.139328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:16.139401 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:16.139554 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:16.139556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:16.139734 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:16.139784 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:16.139786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.140433 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.140435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:16.141051 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:16.141053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.141419 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.141420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:16.141462 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:16.141952 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:16.141957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:16.143951 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
W20260808 00:33:16.144110 13303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:16.139291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:40137 (local address 127.31.210.254:38865)
0808 00:33:16.139344 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:16.139346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:16.139383 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:33:16.139445 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:16.139446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:16.139464 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:16.139516 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:16.139860 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.140357 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.141151 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.141344 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.141403 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:16.144044 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.195:40137: BlockingRecv error: recv got EOF from 127.31.210.195:40137 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 00:33:16 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:16 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.193@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:16.896229 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:16.885616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.193:57375)
0808 00:33:16.885809 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:16.885825 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:16.885904 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:16.886090 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:16.886092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:16.886383 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:16.886451 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:16.886454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.887253 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.887254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:16.887951 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:16.887954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.888397 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.888398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:16.888522 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:16.893807 (+ 5285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:16.893814 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:16.896106 (+ 2292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":81,"threads_started":1}
W20260808 00:33:16.896309 13309 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:16.885738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:57375 (local address 127.31.210.254:38865)
0808 00:33:16.885904 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:16.885906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:16.885918 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:33:16.885973 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:16.885974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:16.885987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:16.886082 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:16.886575 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.887149 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.888067 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:16.888298 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:16.888384 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:16.896220 (+ 7836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.193:57375: BlockingRecv error: recv got EOF from 127.31.210.193:57375 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
Aug 08 00:33:17 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:33:17 dist-test-slave-44d2 krb5kdc[11834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.210.194@KRBTEST.COM for oryx/127.31.210.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:33:17.015219 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.009791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38865 (local address 127.31.210.194:60427)
0808 00:33:17.010007 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:17.010021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:17.010101 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:17.010249 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:17.010250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:17.010463 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:17.010517 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.010518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.011242 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.011243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.011869 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.011885 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.012294 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.012295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.012345 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.013087 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:17.013092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:17.015142 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.210.254:38865: Server oryx/127.31.210.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":131,"threads_started":1}
W20260808 00:33:17.015303 13309 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:17.010006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:60427 (local address 127.31.210.254:38865)
0808 00:33:17.010105 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:17.010107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:17.010120 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:33:17.010152 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:17.010153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:17.010166 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:17.010226 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:17.010605 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.011151 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.012003 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.012215 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.012281 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.015253 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.210.194:60427: BlockingRecv error: recv got EOF from 127.31.210.194:60427 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 00:33:17.051638 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 12654
I20260808 00:33:17.063445 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 12869
I20260808 00:33:17.071756 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 13050
I20260808 00:33:17.081075 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 12553
2026-08-08T00:33:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17895 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T00:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:33:17Z Disabled control of system clock
I20260808 00:33:17.118583 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:35267
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:35267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 00:33:17.201762 13317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:17.201967 13317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:17.201996 13317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:17.203164 13317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:33:17.203207 13317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:17.203219 13317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:17.203231 13317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:33:17.203244 13317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:33:17.204559 13317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:35267
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:35267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:17.204758 13317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:17.205009 13317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:17.210090 13325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.210084 13323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:17.210232 13317 server_base.cc:1034] running on GCE node
W20260808 00:33:17.210119 13328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:17.210605 13317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:17.211040 13317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:17.212245 13317 hybrid_clock.cc:648] HybridClock initialized: now 1786149197212217 us; error 39 us; skew 500 ppm
I20260808 00:33:17.213568 13317 webserver.cc:460] Webserver started at http://127.31.210.254:44935/ using document root <none> and password file <none>
I20260808 00:33:17.213830 13317 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:17.213901 13317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:17.214107 13317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:17.215174 13317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "3a6c3b6fc497365a9f0b7fe120fac661"
server_key_iv: "52f69360579d02c057aa6dfbe1517c68"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.215560 13317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "3a6c3b6fc497365a9f0b7fe120fac661"
server_key_iv: "52f69360579d02c057aa6dfbe1517c68"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.217212 13317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:33:17.218025 13334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.218222 13317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:17.218317 13317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "3a6c3b6fc497365a9f0b7fe120fac661"
server_key_iv: "52f69360579d02c057aa6dfbe1517c68"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.218397 13317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:17.236549 13317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:17.236917 13317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:17.237087 13317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:17.241812 13317 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:35267
I20260808 00:33:17.241850 13399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:35267 every 8 connection(s)
I20260808 00:33:17.242245 13317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:33:17.243023 13400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:33:17.243433 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 13317
I20260808 00:33:17.243531 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:33:17.243747 32587 external_mini_cluster.cc:1428] Setting key 10461145eebd1c70b52155cb0ad0ec4b
I20260808 00:33:17.247342 13400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3: Bootstrap starting.
I20260808 00:33:17.248042 13400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3: Neither blocks nor log segments found. Creating new log.
I20260808 00:33:17.248387 13400 log.cc:824] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:17.249368 13400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3: No bootstrap required, opened a new log
I20260808 00:33:17.250644 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.244390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.31.210.254:35267)
0808 00:33:17.244673 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:17.244683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:17.244719 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:33:17.245295 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:17.245305 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:17.245406 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:17.245575 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:17.245816 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.247370 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.248282 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.248909 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.249075 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.249852 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:17.249857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:17.249857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:17.250091 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:17.250136 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:17.250153 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:17.250232 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:17.250314 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:33:17.250405 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":133,"threads_started":1}
I20260808 00:33:17.252022 13400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35267 } }
I20260808 00:33:17.252168 13400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:33:17.252193 13400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bcfc8ae98eb4b49b56a407365d9e0e3, State: Initialized, Role: FOLLOWER
I20260808 00:33:17.252342 13400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35267 } }
I20260808 00:33:17.252439 13400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:33:17.252465 13400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:33:17.252557 13400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:33:17.253350 13400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35267 } }
I20260808 00:33:17.253487 13400 leader_election.cc:302] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bcfc8ae98eb4b49b56a407365d9e0e3; no voters:
I20260808 00:33:17.253672 13400 leader_election.cc:288] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:33:17.253726 13408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:33:17.253934 13408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8bcfc8ae98eb4b49b56a407365d9e0e3, State: Running, Role: LEADER
I20260808 00:33:17.253988 13400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:33:17.254118 13408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35267 } }
I20260808 00:33:17.254523 13409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35267 } } }
I20260808 00:33:17.254524 13410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bcfc8ae98eb4b49b56a407365d9e0e3. Latest consensus state: current_term: 1 leader_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcfc8ae98eb4b49b56a407365d9e0e3" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 35267 } } }
I20260808 00:33:17.254712 13409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:17.254722 13410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:17.254994 13421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:33:17.255549 13421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:33:17.257368 13421 catalog_manager.cc:1347] Generated new cluster ID: c47fcfa2c5ce41c98c430f8a3c9c7fc4
I20260808 00:33:17.257426 13421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:33:17.266011 13421 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:33:17.266841 13421 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:33:17.276034 13421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bcfc8ae98eb4b49b56a407365d9e0e3: Generated new TSK 0
I20260808 00:33:17.276288 13421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:33:17.283923 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35267
--builtin_ntp_servers=127.31.210.212:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 00:33:17.382812 13435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:17.383065 13435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:17.383106 13435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:17.384328 13435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:17.384413 13435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:17.384459 13435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:33:17.385869 13435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:17.386152 13435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:17.386498 13435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:17.392621 13445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.392726 13443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.392624 13447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:17.393298 13435 server_base.cc:1034] running on GCE node
I20260808 00:33:17.393565 13435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:17.393843 13435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:17.395056 13435 hybrid_clock.cc:648] HybridClock initialized: now 1786149197395014 us; error 56 us; skew 500 ppm
I20260808 00:33:17.396658 13435 webserver.cc:460] Webserver started at http://127.31.210.193:43529/ using document root <none> and password file <none>
I20260808 00:33:17.396942 13435 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:17.396996 13435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:17.397154 13435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:17.398262 13435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "93f52d15894648f3844064c9305af617"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "2dabbe64b5cd566f0b33bfbe7cd9430e"
server_key_iv: "9b360c624c378f067c680abfa73d04ac"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.398653 13435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "93f52d15894648f3844064c9305af617"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "2dabbe64b5cd566f0b33bfbe7cd9430e"
server_key_iv: "9b360c624c378f067c680abfa73d04ac"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.400009 13435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:17.400806 13455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.401013 13435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:17.401119 13435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "93f52d15894648f3844064c9305af617"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "2dabbe64b5cd566f0b33bfbe7cd9430e"
server_key_iv: "9b360c624c378f067c680abfa73d04ac"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.401196 13435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:17.419869 13435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:17.420213 13435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:17.420375 13435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:17.420693 13435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:17.421061 13435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:17.421119 13435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.421156 13435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:17.421187 13435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.427585 13435 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:42617
I20260808 00:33:17.427641 13605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:42617 every 8 connection(s)
I20260808 00:33:17.427964 13435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:33:17.432287 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 13435
I20260808 00:33:17.432386 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:33:17.432659 32587 external_mini_cluster.cc:1428] Setting key 0781944e9fe77c452119959456f36924
I20260808 00:33:17.433902 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35267
--builtin_ntp_servers=127.31.210.212:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 00:33:17.434283 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.428800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:40121 (local address 127.31.210.254:35267)
0808 00:33:17.428854 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:17.428857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:17.429207 (+ 350us) server_negotiation.cc:407] Connection header received
0808 00:33:17.429697 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:17.429699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:17.429716 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:17.429785 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:17.430510 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.431011 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.431962 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.432288 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.432370 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.433911 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:17.433913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:17.433913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:17.433920 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:17.433940 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:17.433945 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:17.434011 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:17.434137 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 00:33:17.434170 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:33:17.434473 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.428784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:35267 (local address 127.31.210.193:40121)
0808 00:33:17.429076 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:17.429110 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:17.429599 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:17.429844 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:17.429847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:17.429856 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:17.430392 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.430397 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.431106 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.431107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.431882 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.431883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.432436 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.432438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.432640 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.433681 (+ 1041us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:33:17.433682 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:17.433724 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:33:17.433752 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:17.434024 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:17.434031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:17.434035 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:33:17.434124 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:33:17.434240 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":184,"threads_started":1}
I20260808 00:33:17.434988 13606 heartbeater.cc:347] Connected to a master server at 127.31.210.254:35267
I20260808 00:33:17.435096 13606 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:17.435307 13606 heartbeater.cc:511] Master 127.31.210.254:35267 requested a full tablet report, sending...
I20260808 00:33:17.435933 13353 ts_manager.cc:194] Registered new tserver with Master: 93f52d15894648f3844064c9305af617 (127.31.210.193:42617)
I20260808 00:33:17.436946 13353 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.193:40121
W20260808 00:33:17.518900 13610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:17.519116 13610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:17.519138 13610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:17.520233 13610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:17.520267 13610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:17.520292 13610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:33:17.521605 13610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:17.521798 13610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:17.522050 13610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:17.527240 13627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.527328 13623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.527269 13619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:17.527529 13610 server_base.cc:1034] running on GCE node
I20260808 00:33:17.527771 13610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:17.528023 13610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:17.529187 13610 hybrid_clock.cc:648] HybridClock initialized: now 1786149197529163 us; error 43 us; skew 500 ppm
I20260808 00:33:17.530496 13610 webserver.cc:460] Webserver started at http://127.31.210.194:38137/ using document root <none> and password file <none>
I20260808 00:33:17.530771 13610 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:17.530848 13610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:17.531049 13610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:17.532167 13610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "9330a77a814544e0a68ce35207787bf6"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "a954dd57d311ee9e2ebb9eb8c9b07cff"
server_key_iv: "05187db2a696a9ca927fd67c5a602ee3"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.532618 13610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "9330a77a814544e0a68ce35207787bf6"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "a954dd57d311ee9e2ebb9eb8c9b07cff"
server_key_iv: "05187db2a696a9ca927fd67c5a602ee3"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.534005 13610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:17.534859 13640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.535068 13610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:17.535176 13610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "9330a77a814544e0a68ce35207787bf6"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "a954dd57d311ee9e2ebb9eb8c9b07cff"
server_key_iv: "05187db2a696a9ca927fd67c5a602ee3"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.535265 13610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:17.556008 13610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:17.556386 13610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:17.556591 13610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:17.556851 13610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:17.557241 13610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:17.557300 13610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.557349 13610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:17.557384 13610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.563253 13610 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:38699
I20260808 00:33:17.563298 13790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:38699 every 8 connection(s)
I20260808 00:33:17.563737 13610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:33:17.569494 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.564635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:46717 (local address 127.31.210.254:35267)
0808 00:33:17.564696 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:17.564698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:17.565094 (+ 396us) server_negotiation.cc:407] Connection header received
0808 00:33:17.565720 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:17.565721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:17.565734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:17.565789 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:17.566515 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.567017 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.568021 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.568219 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.568308 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.569182 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:17.569184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:17.569184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:17.569189 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:17.569209 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:17.569213 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:17.569274 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:17.569383 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 00:33:17.569411 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:33:17.569715 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.564640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:35267 (local address 127.31.210.194:46717)
0808 00:33:17.564848 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:17.564983 (+ 135us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:17.565600 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:17.565882 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:17.565885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:17.565893 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:17.566409 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.566413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.567127 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.567128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.567927 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.567929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.568319 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.568320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.568458 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.569038 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:33:17.569038 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:17.569067 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:33:17.569090 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:17.569276 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:17.569283 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:17.569286 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:33:17.569377 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:33:17.569485 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260808 00:33:17.570380 13793 heartbeater.cc:347] Connected to a master server at 127.31.210.254:35267
I20260808 00:33:17.570538 13793 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:17.570590 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 13610
I20260808 00:33:17.570683 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:33:17.570762 13793 heartbeater.cc:511] Master 127.31.210.254:35267 requested a full tablet report, sending...
I20260808 00:33:17.570895 32587 external_mini_cluster.cc:1428] Setting key 837ef77df93bc4b40491b492e39a56d5
I20260808 00:33:17.571233 13353 ts_manager.cc:194] Registered new tserver with Master: 9330a77a814544e0a68ce35207787bf6 (127.31.210.194:38699)
I20260808 00:33:17.571847 13353 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.194:46717
I20260808 00:33:17.572098 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35267
--builtin_ntp_servers=127.31.210.212:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 00:33:17.657642 13798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:17.657833 13798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:17.657856 13798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:17.658982 13798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:17.659024 13798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:17.659049 13798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:33:17.660377 13798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:35267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:17.660682 13798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:17.661000 13798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:17.666054 13807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.666136 13808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.666098 13811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:17.666224 13798 server_base.cc:1034] running on GCE node
I20260808 00:33:17.666458 13798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:17.666661 13798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:17.667826 13798 hybrid_clock.cc:648] HybridClock initialized: now 1786149197667813 us; error 28 us; skew 500 ppm
I20260808 00:33:17.669297 13798 webserver.cc:460] Webserver started at http://127.31.210.195:44127/ using document root <none> and password file <none>
I20260808 00:33:17.669569 13798 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:17.669622 13798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:17.669807 13798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:17.670948 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "392e3b7546f94219bbd913d84ea44a8b"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "f8794a89129d176113d615d9efb9e7f0"
server_key_iv: "71624d6cf0b62813788cff3a9552137b"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.671336 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "392e3b7546f94219bbd913d84ea44a8b"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "f8794a89129d176113d615d9efb9e7f0"
server_key_iv: "71624d6cf0b62813788cff3a9552137b"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.672833 13798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:17.673580 13817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.673774 13798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:33:17.673873 13798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "392e3b7546f94219bbd913d84ea44a8b"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "f8794a89129d176113d615d9efb9e7f0"
server_key_iv: "71624d6cf0b62813788cff3a9552137b"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.673950 13798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:17.689729 13798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:17.690032 13798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:17.690192 13798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:17.690424 13798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:17.690755 13798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:17.690790 13798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.690811 13798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:17.690860 13798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.696784 13798 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:35613
I20260808 00:33:17.696842 13967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:35613 every 8 connection(s)
I20260808 00:33:17.697168 13798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:33:17.697999 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 13798
I20260808 00:33:17.698119 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:33:17.698374 32587 external_mini_cluster.cc:1428] Setting key d25360a338b73d4b39fc3ff3c593cdda
I20260808 00:33:17.702580 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.697959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:58351 (local address 127.31.210.254:35267)
0808 00:33:17.698038 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:17.698041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:17.698298 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:33:17.698927 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:17.698928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:17.698943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:17.699002 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:17.699823 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.700319 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.701355 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.701540 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.701596 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.702355 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:17.702356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:17.702356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:17.702360 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:17.702375 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:17.702378 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:17.702425 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:17.702501 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 00:33:17.702522 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:33:17.702793 13970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.697928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:35267 (local address 127.31.210.195:58351)
0808 00:33:17.698153 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:17.698194 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:17.698795 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:17.699076 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:17.699081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:17.699092 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:17.699738 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.699744 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.700403 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.700404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.701274 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:17.701276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:17.701615 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:17.701616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:17.701730 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:17.702223 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:33:17.702223 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:17.702250 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:33:17.702273 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:17.702428 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:17.702430 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:17.702432 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:33:17.702484 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:33:17.702561 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":126,"threads_started":1}
I20260808 00:33:17.703384 13968 heartbeater.cc:347] Connected to a master server at 127.31.210.254:35267
I20260808 00:33:17.703500 13968 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:17.703675 13968 heartbeater.cc:511] Master 127.31.210.254:35267 requested a full tablet report, sending...
I20260808 00:33:17.704116 13353 ts_manager.cc:194] Registered new tserver with Master: 392e3b7546f94219bbd913d84ea44a8b (127.31.210.195:35613)
I20260808 00:33:17.704525 13353 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.195:58351
I20260808 00:33:17.710114 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:33:17.711990 13404 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:17.711445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.31.210.254:35267)
0808 00:33:17.711524 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:17.711526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:17.711609 (+ 83us) server_negotiation.cc:407] Connection header received
0808 00:33:17.711644 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:17.711645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:17.711653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:17.711701 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:17.711908 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37694: BlockingRecv error: recv got EOF from 127.0.0.1:37694 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 00:33:17.712006 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 13435
I20260808 00:33:17.723716 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 13610
I20260808 00:33:17.734377 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 13798
I20260808 00:33:17.744081 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 13317
2026-08-08T00:33:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (660 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T00:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:33:17Z Disabled control of system clock
I20260808 00:33:17.778931 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:37597
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:37597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:33:17.863272 13986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:17.863510 13986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:17.863533 13986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:17.864739 13986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:33:17.864782 13986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:17.864796 13986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:17.864809 13986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:33:17.864850 13986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:33:17.866196 13986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:37597
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:37597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:17.866469 13986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:17.866782 13986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:17.871803 13991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:17.871917 13992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:17.871987 13986 server_base.cc:1034] running on GCE node
W20260808 00:33:17.871819 13996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:17.872391 13986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:17.872745 13986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:17.873924 13986 hybrid_clock.cc:648] HybridClock initialized: now 1786149197873885 us; error 51 us; skew 500 ppm
I20260808 00:33:17.875262 13986 webserver.cc:460] Webserver started at http://127.31.210.254:46689/ using document root <none> and password file <none>
I20260808 00:33:17.875551 13986 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:17.875631 13986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:17.875784 13986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:17.876956 13986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "2fe790932d6bd03de993ea426d3676f9"
server_key_iv: "57d731f09dc85ef5e2ae5f38a3cd34fe"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.877321 13986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "2fe790932d6bd03de993ea426d3676f9"
server_key_iv: "57d731f09dc85ef5e2ae5f38a3cd34fe"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.878665 13986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:33:17.879386 14005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:17.879572 13986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:33:17.879683 13986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13"
format_stamp: "Formatted at 2026-08-08 00:33:17 on dist-test-slave-44d2"
server_key: "2fe790932d6bd03de993ea426d3676f9"
server_key_iv: "57d731f09dc85ef5e2ae5f38a3cd34fe"
server_key_version: "encryptionkey@0"
I20260808 00:33:17.879760 13986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:17.883229 13986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:17.883512 13986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:17.883661 13986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:17.887676 13986 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:37597
I20260808 00:33:17.887706 14071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:37597 every 8 connection(s)
I20260808 00:33:17.888036 13986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:33:17.888820 14072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:33:17.891505 14072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13: Bootstrap starting.
I20260808 00:33:17.892032 14072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13: Neither blocks nor log segments found. Creating new log.
I20260808 00:33:17.892283 14072 log.cc:824] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:17.893033 14072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13: No bootstrap required, opened a new log
I20260808 00:33:17.893986 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 13986
I20260808 00:33:17.894131 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:33:17.894357 32587 external_mini_cluster.cc:1428] Setting key 05cdbab90741fa17c3b9c068471c5cd3
I20260808 00:33:17.895044 14072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37597 } }
I20260808 00:33:17.895210 14072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:33:17.895263 14072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161d7b1dcfdb48ffb6c9cbb85db57a13, State: Initialized, Role: FOLLOWER
I20260808 00:33:17.895392 14072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37597 } }
I20260808 00:33:17.895485 14072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:33:17.895517 14072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:33:17.895562 14072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:33:17.896240 14072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37597 } }
I20260808 00:33:17.896374 14072 leader_election.cc:302] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 161d7b1dcfdb48ffb6c9cbb85db57a13; no voters:
I20260808 00:33:17.896585 14072 leader_election.cc:288] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:33:17.896651 14077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:33:17.896796 14077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [term 1 LEADER]: Becoming Leader. State: Replica: 161d7b1dcfdb48ffb6c9cbb85db57a13, State: Running, Role: LEADER
I20260808 00:33:17.896906 14077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37597 } }
I20260808 00:33:17.896911 14072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:33:17.897270 14077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161d7b1dcfdb48ffb6c9cbb85db57a13. Latest consensus state: current_term: 1 leader_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37597 } } }
I20260808 00:33:17.897280 14079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161d7b1dcfdb48ffb6c9cbb85db57a13" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37597 } } }
I20260808 00:33:17.897421 14077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:17.897500 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:17.895042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.31.210.254:37597)
0808 00:33:17.895342 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:17.895351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:17.895384 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:33:17.895501 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:17.895510 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:17.895555 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:17.895706 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:17.896507 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:17.896512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:17.896513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:17.896518 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:17.896570 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:17.896586 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:17.896667 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:17.896812 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 00:33:17.896873 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":109,"threads_started":1}
I20260808 00:33:17.897790 14079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13 [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:17.898196 14089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:33:17.898759 14089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:33:17.900346 14089 catalog_manager.cc:1347] Generated new cluster ID: bcb83e4f845d48afac610bf6732d0f12
I20260808 00:33:17.900422 14089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:33:17.910619 14089 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:33:17.910808 14089 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:33:17.921881 14089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 161d7b1dcfdb48ffb6c9cbb85db57a13: Generated new TSK 0
I20260808 00:33:17.922149 14089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:33:17.928704 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:37597
--builtin_ntp_servers=127.31.210.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:33:18.015604 14104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:18.015820 14104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:18.015842 14104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:18.017115 14104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:18.017159 14104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:18.017186 14104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:33:18.018714 14104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:18.018986 14104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:18.019345 14104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:18.024776 14110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:18.024825 14109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:18.024974 14116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:18.025033 14104 server_base.cc:1034] running on GCE node
I20260808 00:33:18.025319 14104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:18.025580 14104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:18.026753 14104 hybrid_clock.cc:648] HybridClock initialized: now 1786149198026739 us; error 31 us; skew 500 ppm
I20260808 00:33:18.028081 14104 webserver.cc:460] Webserver started at http://127.31.210.193:33509/ using document root <none> and password file <none>
I20260808 00:33:18.028337 14104 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:18.028409 14104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:18.028620 14104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:18.029716 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "a8a20da281714c16b113a8b2574fb1d5"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "163b2af553922897681f1e6538cd9e13"
server_key_iv: "a69611ebc1e7dc25cdde8f67d68fe786"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.030095 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "a8a20da281714c16b113a8b2574fb1d5"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "163b2af553922897681f1e6538cd9e13"
server_key_iv: "a69611ebc1e7dc25cdde8f67d68fe786"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.031517 14104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:18.032335 14128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.032590 14104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:18.032708 14104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "a8a20da281714c16b113a8b2574fb1d5"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "163b2af553922897681f1e6538cd9e13"
server_key_iv: "a69611ebc1e7dc25cdde8f67d68fe786"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.032797 14104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:18.036259 14104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:18.036581 14104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:18.036734 14104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:18.036947 14104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:18.037267 14104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:18.037328 14104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.037369 14104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:18.037395 14104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.043190 14104 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:39367
I20260808 00:33:18.043233 14281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:39367 every 8 connection(s)
I20260808 00:33:18.043603 14104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:33:18.045086 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 14104
I20260808 00:33:18.045188 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:33:18.045387 32587 external_mini_cluster.cc:1428] Setting key 3c1100df79b802bd4235344f12e7b439
I20260808 00:33:18.046631 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:37597
--builtin_ntp_servers=127.31.210.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 00:33:18.047405 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:18.044406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:56555 (local address 127.31.210.254:37597)
0808 00:33:18.044527 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:18.044529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:18.044727 (+ 198us) server_negotiation.cc:407] Connection header received
0808 00:33:18.045440 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:18.045441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:18.045448 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:18.045532 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:18.047123 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:18.047125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:18.047125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:18.047129 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:18.047151 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:18.047156 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:18.047215 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:18.047330 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 00:33:18.047346 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:33:18.047643 14284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:18.044359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:37597 (local address 127.31.210.193:56555)
0808 00:33:18.044581 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:18.044625 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:18.045318 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:18.045608 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:18.045613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:18.045624 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:18.046910 (+ 1286us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:33:18.046911 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:18.046950 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:33:18.046988 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:18.047233 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:18.047240 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:18.047244 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:33:18.047330 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 00:33:18.047403 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260808 00:33:18.048352 14282 heartbeater.cc:347] Connected to a master server at 127.31.210.254:37597
I20260808 00:33:18.048537 14282 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:18.048727 14282 heartbeater.cc:511] Master 127.31.210.254:37597 requested a full tablet report, sending...
I20260808 00:33:18.049276 14025 ts_manager.cc:194] Registered new tserver with Master: a8a20da281714c16b113a8b2574fb1d5 (127.31.210.193:39367)
W20260808 00:33:18.131944 14285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:18.132164 14285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:18.132189 14285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:18.133416 14285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:18.133466 14285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:18.133495 14285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:33:18.134876 14285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:18.135093 14285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:18.135460 14285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:18.140939 14297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:18.140972 14292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:18.141098 14285 server_base.cc:1034] running on GCE node
W20260808 00:33:18.140973 14294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:18.141336 14285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:18.141614 14285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:18.142760 14285 hybrid_clock.cc:648] HybridClock initialized: now 1786149198142751 us; error 32 us; skew 500 ppm
I20260808 00:33:18.144047 14285 webserver.cc:460] Webserver started at http://127.31.210.194:40055/ using document root <none> and password file <none>
I20260808 00:33:18.144295 14285 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:18.144345 14285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:18.144562 14285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:18.145656 14285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "0c191adc50fc4a76b18d60d90473b607"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "c8d47143ad9dd23f9119f3b1cb67aade"
server_key_iv: "240cb92514457477dc4b7d56b161af7e"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.146020 14285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "0c191adc50fc4a76b18d60d90473b607"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "c8d47143ad9dd23f9119f3b1cb67aade"
server_key_iv: "240cb92514457477dc4b7d56b161af7e"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.147330 14285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:33:18.148073 14308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.148247 14285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:33:18.148351 14285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "0c191adc50fc4a76b18d60d90473b607"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "c8d47143ad9dd23f9119f3b1cb67aade"
server_key_iv: "240cb92514457477dc4b7d56b161af7e"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.148432 14285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:18.151808 14285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:18.152063 14285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:18.152210 14285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:18.152441 14285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:18.152796 14285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:18.152851 14285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.152892 14285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:18.152920 14285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.158684 14285 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:44205
I20260808 00:33:18.158720 14471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:44205 every 8 connection(s)
I20260808 00:33:18.159082 14285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:33:18.162173 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:18.159884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:36927 (local address 127.31.210.254:37597)
0808 00:33:18.159954 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:18.159956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:18.160235 (+ 279us) server_negotiation.cc:407] Connection header received
0808 00:33:18.160771 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:18.160773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:18.160779 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:18.160837 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:18.161938 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:18.161939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:18.161940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:18.161944 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:18.161962 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:18.161967 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:18.162016 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:18.162106 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 00:33:18.162119 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:33:18.162338 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:18.159870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:37597 (local address 127.31.210.194:36927)
0808 00:33:18.160112 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:18.160146 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:18.160668 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:18.160892 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:18.160895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:18.160904 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:18.161798 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:33:18.161798 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:18.161825 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:33:18.161854 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:18.162036 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:18.162039 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:18.162042 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:33:18.162097 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:33:18.162145 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":153,"threads_started":1}
I20260808 00:33:18.162689 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 14285
I20260808 00:33:18.162812 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:33:18.163021 32587 external_mini_cluster.cc:1428] Setting key e2fe5b6987b7f815bb33d99be14d80f4
I20260808 00:33:18.163120 14472 heartbeater.cc:347] Connected to a master server at 127.31.210.254:37597
I20260808 00:33:18.163232 14472 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:18.163404 14472 heartbeater.cc:511] Master 127.31.210.254:37597 requested a full tablet report, sending...
I20260808 00:33:18.163862 14025 ts_manager.cc:194] Registered new tserver with Master: 0c191adc50fc4a76b18d60d90473b607 (127.31.210.194:44205)
I20260808 00:33:18.164227 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:37597
--builtin_ntp_servers=127.31.210.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:33:18.251395 14476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:18.251648 14476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:18.251674 14476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:18.252981 14476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:18.253022 14476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:18.253048 14476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:33:18.254395 14476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:18.254637 14476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:18.254944 14476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:18.260210 14486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:18.260247 14490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:18.260210 14488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:18.260524 14476 server_base.cc:1034] running on GCE node
I20260808 00:33:18.260826 14476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:18.261173 14476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:18.262374 14476 hybrid_clock.cc:648] HybridClock initialized: now 1786149198262368 us; error 45 us; skew 500 ppm
I20260808 00:33:18.263687 14476 webserver.cc:460] Webserver started at http://127.31.210.195:35623/ using document root <none> and password file <none>
I20260808 00:33:18.263953 14476 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:18.264029 14476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:18.264181 14476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:18.265344 14476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "e75cea2e9cf549e0adae16f4c45fdf3c"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "1a92699e983e059bfa38c19f1013c40b"
server_key_iv: "1e21733f38bdddcada84048affd1ed76"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.265722 14476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "e75cea2e9cf549e0adae16f4c45fdf3c"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "1a92699e983e059bfa38c19f1013c40b"
server_key_iv: "1e21733f38bdddcada84048affd1ed76"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.267130 14476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:33:18.267851 14499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.268024 14476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:33:18.268116 14476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "e75cea2e9cf549e0adae16f4c45fdf3c"
format_stamp: "Formatted at 2026-08-08 00:33:18 on dist-test-slave-44d2"
server_key: "1a92699e983e059bfa38c19f1013c40b"
server_key_iv: "1e21733f38bdddcada84048affd1ed76"
server_key_version: "encryptionkey@0"
I20260808 00:33:18.268201 14476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:18.271582 14476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:18.271862 14476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:18.272010 14476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:18.272243 14476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:18.272630 14476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:18.272686 14476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.272725 14476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:18.272753 14476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:18.280963 14476 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:37969
I20260808 00:33:18.281020 14660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:37969 every 8 connection(s)
I20260808 00:33:18.281400 14476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:33:18.284919 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:18.282220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:35063 (local address 127.31.210.254:37597)
0808 00:33:18.282267 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:18.282269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:18.282543 (+ 274us) server_negotiation.cc:407] Connection header received
0808 00:33:18.283153 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:18.283155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:18.283162 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:18.283237 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:18.284613 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:18.284615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:18.284615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:33:18.284621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:18.284637 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:33:18.284643 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:18.284707 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:18.284819 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:33:18.284839 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:33:18.285127 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:18.282200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:37597 (local address 127.31.210.195:35063)
0808 00:33:18.282396 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:18.282450 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:18.283021 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:18.283335 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:18.283338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:18.283347 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:18.284387 (+ 1040us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:33:18.284387 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:18.284416 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:33:18.284444 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:18.284723 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:18.284729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:18.284732 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:33:18.284829 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 00:33:18.284902 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260808 00:33:18.285599 14661 heartbeater.cc:347] Connected to a master server at 127.31.210.254:37597
I20260808 00:33:18.285686 14661 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:18.285892 14661 heartbeater.cc:511] Master 127.31.210.254:37597 requested a full tablet report, sending...
I20260808 00:33:18.286267 14025 ts_manager.cc:194] Registered new tserver with Master: e75cea2e9cf549e0adae16f4c45fdf3c (127.31.210.195:37969)
I20260808 00:33:18.290326 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 14476
I20260808 00:33:18.290481 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:33:18.290750 32587 external_mini_cluster.cc:1428] Setting key 30b843b4b2142fb1d012ebb53a39ee21
I20260808 00:33:18.291932 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:33:18.294086 14076 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:33:18.293457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.31.210.254:37597)
0808 00:33:18.293525 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:18.293528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:18.293676 (+ 148us) server_negotiation.cc:407] Connection header received
0808 00:33:18.293708 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:18.293709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:18.293715 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:18.293789 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:18.293949 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34536: BlockingRecv error: recv got EOF from 127.0.0.1:34536 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:33:18.294121 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 14104
I20260808 00:33:18.322793 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 14285
I20260808 00:33:18.341327 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 14476
I20260808 00:33:18.349181 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 13986
2026-08-08T00:33:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (604 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:33:18 dist-test-slave-44d2 krb5kdc[14676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:33:18 dist-test-slave-44d2 krb5kdc[14676](info): set up 2 sockets
Aug 08 00:33:18 dist-test-slave-44d2 krb5kdc[14676](info): commencing operation
krb5kdc: starting...
W20260808 00:33:19.585106 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:33:19 dist-test-slave-44d2 krb5kdc[14676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:33:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:33:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:33:19.717461 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:45141
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:45141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:19.801429 14694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:19.801657 14694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:19.801680 14694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:19.802979 14694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:33:19.803030 14694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:19.803045 14694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:19.803057 14694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:33:19.803069 14694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:33:19.805171 14694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:45141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:45141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:19.805446 14694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:19.805708 14694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:19.810750 14704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:19.810755 14705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:19.810798 14707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:19.811082 14694 server_base.cc:1034] running on GCE node
I20260808 00:33:19.811359 14694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:19.811705 14694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:19.812877 14694 hybrid_clock.cc:648] HybridClock initialized: now 1786149199812861 us; error 27 us; skew 500 ppm
Aug 08 00:33:19 dist-test-slave-44d2 krb5kdc[14676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149199, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:19.815389 14694 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:33:19.815935 14694 webserver.cc:460] Webserver started at http://127.31.210.254:42597/ using document root <none> and password file <none>
I20260808 00:33:19.816195 14694 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:19.816267 14694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:19.816428 14694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:19.817572 14694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "4e3204b1184a4305bec64dea0290f6c8"
format_stamp: "Formatted at 2026-08-08 00:33:19 on dist-test-slave-44d2"
server_key: "67aac9d4e471a93e0c021b4dce6f976a"
server_key_iv: "a62e3fe85dce2546ff04eb67fc2843ed"
server_key_version: "encryptionkey@0"
I20260808 00:33:19.817958 14694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "4e3204b1184a4305bec64dea0290f6c8"
format_stamp: "Formatted at 2026-08-08 00:33:19 on dist-test-slave-44d2"
server_key: "67aac9d4e471a93e0c021b4dce6f976a"
server_key_iv: "a62e3fe85dce2546ff04eb67fc2843ed"
server_key_version: "encryptionkey@0"
I20260808 00:33:19.819540 14694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:19.820369 14717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:19.820619 14694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:33:19.820719 14694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "4e3204b1184a4305bec64dea0290f6c8"
format_stamp: "Formatted at 2026-08-08 00:33:19 on dist-test-slave-44d2"
server_key: "67aac9d4e471a93e0c021b4dce6f976a"
server_key_iv: "a62e3fe85dce2546ff04eb67fc2843ed"
server_key_version: "encryptionkey@0"
I20260808 00:33:19.820806 14694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:19.837575 14694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:19.839610 14694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:19.839784 14694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:19.844209 14694 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:45141
I20260808 00:33:19.844245 14789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:45141 every 8 connection(s)
I20260808 00:33:19.844645 14694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:33:19.845682 14792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:33:19.847958 14792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8: Bootstrap starting.
I20260808 00:33:19.848623 14792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8: Neither blocks nor log segments found. Creating new log.
I20260808 00:33:19.848868 14792 log.cc:824] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:19.849619 14792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8: No bootstrap required, opened a new log
I20260808 00:33:19.851402 14792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3204b1184a4305bec64dea0290f6c8" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45141 } }
I20260808 00:33:19.851555 14792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:33:19.851581 14792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3204b1184a4305bec64dea0290f6c8, State: Initialized, Role: FOLLOWER
I20260808 00:33:19.851697 14792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3204b1184a4305bec64dea0290f6c8" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45141 } }
I20260808 00:33:19.851800 14792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:33:19.851840 14792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:33:19.851913 14792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:33:19.852897 14792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3204b1184a4305bec64dea0290f6c8" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45141 } }
I20260808 00:33:19.853050 14792 leader_election.cc:302] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e3204b1184a4305bec64dea0290f6c8; no voters:
I20260808 00:33:19.853199 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 14694
I20260808 00:33:19.853245 14792 leader_election.cc:288] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:33:19.853286 14796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:33:19.853312 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:33:19.853483 14796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [term 1 LEADER]: Becoming Leader. State: Replica: 4e3204b1184a4305bec64dea0290f6c8, State: Running, Role: LEADER
I20260808 00:33:19.853578 32587 external_mini_cluster.cc:1428] Setting key 4d80e3fece5b831426283167e445bd40
I20260808 00:33:19.853529 14792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:33:19.853662 14796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3204b1184a4305bec64dea0290f6c8" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45141 } }
I20260808 00:33:19.854305 14798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e3204b1184a4305bec64dea0290f6c8. Latest consensus state: current_term: 1 leader_uuid: "4e3204b1184a4305bec64dea0290f6c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3204b1184a4305bec64dea0290f6c8" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45141 } } }
I20260808 00:33:19.854513 14798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:19.854813 14797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e3204b1184a4305bec64dea0290f6c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3204b1184a4305bec64dea0290f6c8" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 45141 } } }
I20260808 00:33:19.854965 14797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8 [sys.catalog]: This master's current role is: LEADER
I20260808 00:33:19.855281 14811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:33:19.856040 14811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:33:19.857828 14811 catalog_manager.cc:1347] Generated new cluster ID: ea41dddd465540b0a2d20800f35af41a
I20260808 00:33:19.857885 14811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:33:19 dist-test-slave-44d2 krb5kdc[14676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:19.865217 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:19.854083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.31.210.254:45141)
0808 00:33:19.854429 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:19.854438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:19.854486 (+ 48us) server_negotiation.cc:407] Connection header received
0808 00:33:19.854610 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:19.854617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:19.854680 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:19.855478 (+ 798us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:19.855689 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:19.857025 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:19.858182 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:19.858612 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:19.858757 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:19.861464 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:19.861468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:19.861469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:19.861481 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:19.863378 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:19.863807 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:19.863811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:19.863811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:19.863932 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:19.864185 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:19.864186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:19.864187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:19.864473 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:19.864730 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:19.864808 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 00:33:19.864973 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":84,"threads_started":1}
I20260808 00:33:19.872335 14811 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:33:19.873111 14811 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:33:19.877242 14811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8: Generated new TSK 0
I20260808 00:33:19.877437 14811 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:33:19.912565 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:45141
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:19.999413 14824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:19.999626 14824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:19.999651 14824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:20.000947 14824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:20.001003 14824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:20.001032 14824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:33:20.002553 14824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:20.002841 14824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:20.003137 14824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:20.008571 14831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:20.008584 14830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:20.008702 14824 server_base.cc:1034] running on GCE node
W20260808 00:33:20.008787 14833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:20.008991 14824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:20.009268 14824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:20.010443 14824 hybrid_clock.cc:648] HybridClock initialized: now 1786149200010438 us; error 37 us; skew 500 ppm
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[14676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:20.013103 14824 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:33:20.013587 14824 webserver.cc:460] Webserver started at http://127.31.210.193:36517/ using document root <none> and password file <none>
I20260808 00:33:20.013845 14824 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:20.013927 14824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:20.014083 14824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:20.015153 14824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "f5608ba5c2fa4b4397156988237fd208"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "762f1650406026e99baf6ecb9e143557"
server_key_iv: "400cf766c0661a3f5293659c8567ba66"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.015522 14824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "f5608ba5c2fa4b4397156988237fd208"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "762f1650406026e99baf6ecb9e143557"
server_key_iv: "400cf766c0661a3f5293659c8567ba66"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.016919 14824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:20.017668 14842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.017848 14824 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:33:20.017956 14824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "f5608ba5c2fa4b4397156988237fd208"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "762f1650406026e99baf6ecb9e143557"
server_key_iv: "400cf766c0661a3f5293659c8567ba66"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.018034 14824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:20.036175 14824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:20.038233 14824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:20.038415 14824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:20.038691 14824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:20.039016 14824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:20.039073 14824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.039119 14824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:20.039153 14824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.045356 14824 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:41049
I20260808 00:33:20.045404 15006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:41049 every 8 connection(s)
I20260808 00:33:20.045758 14824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:33:20.049064 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 14824
I20260808 00:33:20.049219 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:33:20.049443 32587 external_mini_cluster.cc:1428] Setting key 5c053c7a6a4a0cc3b18544e1b43e1f7d
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[14676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:20.057675 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.046592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:45141 (local address 127.31.210.193:35385)
0808 00:33:20.046777 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.046825 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.047445 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.047691 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.047695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.048341 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:20.048582 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.048587 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.049373 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.049374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.050802 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.050806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.051219 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.051220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.051382 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.052052 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:20.052065 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:20.053989 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:20.056167 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:20.056174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:20.056176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:20.056410 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:20.056698 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:20.056699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:20.056699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:20.056807 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:20.057150 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:20.057155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:20.057272 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.057359 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.057495 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260808 00:33:20.057713 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.046624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:35385 (local address 127.31.210.254:45141)
0808 00:33:20.046698 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.046700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.047005 (+ 305us) server_negotiation.cc:407] Connection header received
0808 00:33:20.047560 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.047561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.047570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.047633 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:20.048673 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.049270 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.050903 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.051117 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.051181 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.054099 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:20.054101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:20.054101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:20.054108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:20.056077 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.056523 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.056525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.056525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.056611 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.056878 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.056879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.056880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.057055 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:20.057160 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.057430 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.057554 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:33:20.058419 15009 heartbeater.cc:347] Connected to a master server at 127.31.210.254:45141
I20260808 00:33:20.058560 15009 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:20.058749 15009 heartbeater.cc:511] Master 127.31.210.254:45141 requested a full tablet report, sending...
I20260808 00:33:20.059429 14740 ts_manager.cc:194] Registered new tserver with Master: f5608ba5c2fa4b4397156988237fd208 (127.31.210.193:41049)
I20260808 00:33:20.060581 14740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:35385
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:33:20.083781 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:45141
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:20.170034 15017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:20.170285 15017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:20.170324 15017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:20.171501 15017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:20.171576 15017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:20.171627 15017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:33:20.173187 15017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:20.173468 15017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:20.173702 15017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:20.178858 15022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:20.178897 15023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:20.179067 15017 server_base.cc:1034] running on GCE node
W20260808 00:33:20.178887 15026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:20.179382 15017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:20.179597 15017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:20.180766 15017 hybrid_clock.cc:648] HybridClock initialized: now 1786149200180750 us; error 29 us; skew 500 ppm
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[14676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:20.183658 15017 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:33:20.184192 15017 webserver.cc:460] Webserver started at http://127.31.210.194:32847/ using document root <none> and password file <none>
I20260808 00:33:20.184433 15017 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:20.184530 15017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:20.184695 15017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:20.185756 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "d66e2de3fdac4379a6e1c8a439e4d182"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "c242fd996cc9d7461136b37f0e0a42eb"
server_key_iv: "24302014c75c39abaab0c8ce2c66bd33"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.186136 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "d66e2de3fdac4379a6e1c8a439e4d182"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "c242fd996cc9d7461136b37f0e0a42eb"
server_key_iv: "24302014c75c39abaab0c8ce2c66bd33"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.187510 15017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:20.188213 15036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.188387 15017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:33:20.188512 15017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "d66e2de3fdac4379a6e1c8a439e4d182"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "c242fd996cc9d7461136b37f0e0a42eb"
server_key_iv: "24302014c75c39abaab0c8ce2c66bd33"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.188599 15017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:20.202046 15017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:20.203926 15017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:20.204098 15017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:20.204324 15017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:20.204743 15017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:20.204800 15017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.204838 15017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:20.204864 15017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.210906 15017 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:38641
I20260808 00:33:20.210958 15191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:38641 every 8 connection(s)
I20260808 00:33:20.211273 15017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[14676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:20.219431 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 15017
I20260808 00:33:20.219555 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:33:20.219780 32587 external_mini_cluster.cc:1428] Setting key e868d7b346e3fd6c3b1c9955242068c1
I20260808 00:33:20.221185 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.212041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:57073 (local address 127.31.210.254:45141)
0808 00:33:20.212104 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.212106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.212328 (+ 222us) server_negotiation.cc:407] Connection header received
0808 00:33:20.212888 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.212889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.212900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.212965 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:20.213942 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.214449 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.215441 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.215648 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.215703 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.217815 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:20.217816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:20.217817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:20.217823 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:20.219279 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.219782 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.219783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.219784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.219885 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.220228 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.220231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.220232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.220387 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:20.220461 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.221019 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.221114 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:33:20.221321 15197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.212007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:45141 (local address 127.31.210.194:57073)
0808 00:33:20.212162 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.212203 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.212776 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.213059 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.213063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.213626 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:20.213844 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.213850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.214553 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.214554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.215363 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.215365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.215722 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.215723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.215819 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.216309 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:20.216317 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:20.217731 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:20.219358 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:20.219366 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:20.219368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:20.219690 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:20.219994 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:20.219995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:20.219995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:20.220141 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:20.220524 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:20.220532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:20.220717 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.220846 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.221049 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260808 00:33:20.221923 15194 heartbeater.cc:347] Connected to a master server at 127.31.210.254:45141
I20260808 00:33:20.222086 15194 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:20.222299 15194 heartbeater.cc:511] Master 127.31.210.254:45141 requested a full tablet report, sending...
I20260808 00:33:20.222754 14740 ts_manager.cc:194] Registered new tserver with Master: d66e2de3fdac4379a6e1c8a439e4d182 (127.31.210.194:38641)
I20260808 00:33:20.223351 14740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:57073
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:33:20.251127 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:45141
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:33:20.334591 15201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:33:20.334872 15201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:33:20.334916 15201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:33:20.336063 15201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:33:20.336135 15201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:33:20.336174 15201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:33:20.337641 15201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:33:20.337929 15201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:33:20.338189 15201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:33:20.343551 15211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:20.343540 15212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:33:20.343535 15215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:33:20.343823 15201 server_base.cc:1034] running on GCE node
I20260808 00:33:20.344075 15201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:33:20.344326 15201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:33:20.345492 15201 hybrid_clock.cc:648] HybridClock initialized: now 1786149200345471 us; error 40 us; skew 500 ppm
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[14676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:33:20.348209 15201 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:33:20.348733 15201 webserver.cc:460] Webserver started at http://127.31.210.195:40149/ using document root <none> and password file <none>
I20260808 00:33:20.348984 15201 fs_manager.cc:359] Metadata directory not provided
I20260808 00:33:20.349056 15201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:33:20.349206 15201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:33:20.350262 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "be0827f76581485f9fc26d69b02e56ef"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "a436117b7510e7d4226c7d61c39f6c39"
server_key_iv: "bdb68deb7e90c956a804b3558cbb8ff7"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.350656 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "be0827f76581485f9fc26d69b02e56ef"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "a436117b7510e7d4226c7d61c39f6c39"
server_key_iv: "bdb68deb7e90c956a804b3558cbb8ff7"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.352075 15201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:33:20.352895 15227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.353107 15201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:33:20.353219 15201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "be0827f76581485f9fc26d69b02e56ef"
format_stamp: "Formatted at 2026-08-08 00:33:20 on dist-test-slave-44d2"
server_key: "a436117b7510e7d4226c7d61c39f6c39"
server_key_iv: "bdb68deb7e90c956a804b3558cbb8ff7"
server_key_version: "encryptionkey@0"
I20260808 00:33:20.353317 15201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:33:20.363901 15201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:33:20.366693 15201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:33:20.366890 15201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:33:20.367169 15201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:33:20.367523 15201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:33:20.367585 15201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.367632 15201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:33:20.367659 15201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:33:20.374447 15201 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:35443
I20260808 00:33:20.374538 15390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:35443 every 8 connection(s)
I20260808 00:33:20.374899 15201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:33:20.377012 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 15201
I20260808 00:33:20.377127 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:33:20.377370 32587 external_mini_cluster.cc:1428] Setting key 8e1c3b515f3acdfe0846574be9b54613
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[14676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149200, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:33:20.385421 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.375724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:57577 (local address 127.31.210.254:45141)
0808 00:33:20.375801 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.375803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.376118 (+ 315us) server_negotiation.cc:407] Connection header received
0808 00:33:20.376752 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.376753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.376765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.376835 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:20.378366 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.378873 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.379990 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.380170 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.380221 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.382545 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:20.382547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:20.382547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:20.382555 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:20.384032 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.384450 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.384451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.384452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.384582 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.384845 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.384847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.384847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.384979 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:20.385045 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.385286 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.385354 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:33:20.385532 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.375696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:45141 (local address 127.31.210.195:57577)
0808 00:33:20.375885 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.376021 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.376651 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.376938 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.376944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.378027 (+ 1083us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:33:20.378245 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.378250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.379021 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.379022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.379898 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.379900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.380254 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.380254 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.380364 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.380899 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:33:20.380908 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:33:20.382419 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:33:20.384128 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:20.384134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:20.384136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:20.384355 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:20.384654 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:33:20.384655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:33:20.384655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:33:20.384760 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:33:20.385045 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:33:20.385050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:33:20.385166 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.385248 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.385371 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260808 00:33:20.386046 15391 heartbeater.cc:347] Connected to a master server at 127.31.210.254:45141
I20260808 00:33:20.386134 15391 heartbeater.cc:464] Registering TS with master...
I20260808 00:33:20.386350 15391 heartbeater.cc:511] Master 127.31.210.254:45141 requested a full tablet report, sending...
I20260808 00:33:20.386732 14740 ts_manager.cc:194] Registered new tserver with Master: be0827f76581485f9fc26d69b02e56ef (127.31.210.195:35443)
I20260808 00:33:20.387107 14740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:57577
I20260808 00:33:20.388984 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:33:20.396380 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.390234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.31.210.254:45141)
0808 00:33:20.390275 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.390277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.390359 (+ 82us) server_negotiation.cc:407] Connection header received
0808 00:33:20.390411 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.390411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.390419 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.390503 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:33:20.391239 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.391705 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.392570 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.392786 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.392872 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.393996 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:33:20.393997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:33:20.393998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:33:20.394005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:33:20.395199 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.395566 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.395567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.395568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.395629 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:33:20.395829 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:33:20.395830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:33:20.395830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:33:20.395984 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:33:20.396094 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.396196 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.396279 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:33:20.398000 14740 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:33:20.399593 14740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:33:20.406028 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.402122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:35443 (local address 127.0.0.1:55784)
0808 00:33:20.402413 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.402424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.402511 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.402739 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.402740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.402742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:33:20.402790 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.402792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.403661 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.403662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.404863 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.404867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.405672 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.405673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.405760 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.405773 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.405875 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.405934 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":12,"thread_start_us":82,"threads_started":1}
I20260808 00:33:20.406028 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.402347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.31.210.195:35443)
0808 00:33:20.402526 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.402530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.402540 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:33:20.402579 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.402581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.402597 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.402671 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:33:20.402871 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.403556 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.404998 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.405583 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.405673 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.405732 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.405894 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.405934 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260808 00:33:20.406332 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.401912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:38641 (local address 127.0.0.1:56778)
0808 00:33:20.402107 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.402150 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.402213 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.402613 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.402617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.402619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:33:20.402682 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.402684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.404105 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.404106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.405157 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.405159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.406106 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.406107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.406172 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.406175 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.406204 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.406243 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":138,"threads_started":1}
I20260808 00:33:20.406771 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.402211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.31.210.194:38641)
0808 00:33:20.402409 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.402414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.402425 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:33:20.402473 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.402483 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.402498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.402795 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:33:20.402892 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.403988 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.405245 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.406006 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.406531 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.406601 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.406629 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.406681 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260808 00:33:20.407022 15411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.401916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:41049 (local address 127.0.0.1:39958)
0808 00:33:20.402314 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.402323 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.402370 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.402720 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.402720 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.402722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:33:20.402770 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.402772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.404587 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.404588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.405699 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.405702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.406752 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.406753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.406827 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.406832 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.406895 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.406942 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":61,"threads_started":1}
I20260808 00:33:20.407074 15291 tablet_service.cc:1467] Processing CreateTablet for tablet 219aed784ab34f8ea770e8c63b7d1e7d (DEFAULT_TABLE table=test-table [id=a9d6b5aecf4c47ec9f13d672c19b7497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:33:20.407297 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.402021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.31.210.193:41049)
0808 00:33:20.402182 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.402187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.402433 (+ 246us) server_negotiation.cc:407] Connection header received
0808 00:33:20.402478 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.402480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.402498 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.402595 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:33:20.402857 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.403973 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.405808 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.406608 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.406674 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.406718 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.406995 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.407041 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260808 00:33:20.407503 15291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219aed784ab34f8ea770e8c63b7d1e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:33:20.407895 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 219aed784ab34f8ea770e8c63b7d1e7d (DEFAULT_TABLE table=test-table [id=a9d6b5aecf4c47ec9f13d672c19b7497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:33:20.407891 15110 tablet_service.cc:1467] Processing CreateTablet for tablet 219aed784ab34f8ea770e8c63b7d1e7d (DEFAULT_TABLE table=test-table [id=a9d6b5aecf4c47ec9f13d672c19b7497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:33:20.408161 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219aed784ab34f8ea770e8c63b7d1e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:33:20.408213 15110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219aed784ab34f8ea770e8c63b7d1e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:33:20.410923 15417 tablet_bootstrap.cc:492] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208: Bootstrap starting.
I20260808 00:33:20.411592 15419 tablet_bootstrap.cc:492] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Bootstrap starting.
I20260808 00:33:20.411672 15417 tablet_bootstrap.cc:654] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208: Neither blocks nor log segments found. Creating new log.
I20260808 00:33:20.411702 15422 tablet_bootstrap.cc:492] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Bootstrap starting.
I20260808 00:33:20.412163 15419 tablet_bootstrap.cc:654] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Neither blocks nor log segments found. Creating new log.
I20260808 00:33:20.412369 15417 log.cc:824] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:20.412447 15422 tablet_bootstrap.cc:654] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Neither blocks nor log segments found. Creating new log.
I20260808 00:33:20.412843 15419 log.cc:824] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:20.413097 15422 log.cc:824] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Log is configured to *not* fsync() on all Append() calls
I20260808 00:33:20.413338 15417 tablet_bootstrap.cc:492] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208: No bootstrap required, opened a new log
I20260808 00:33:20.413429 15417 ts_tablet_manager.cc:1397] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:33:20.413811 15419 tablet_bootstrap.cc:492] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: No bootstrap required, opened a new log
I20260808 00:33:20.413854 15422 tablet_bootstrap.cc:492] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: No bootstrap required, opened a new log
I20260808 00:33:20.413899 15419 ts_tablet_manager.cc:1397] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:33:20.413919 15422 ts_tablet_manager.cc:1397] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:33:20.415248 15419 raft_consensus.cc:348] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.415403 15419 raft_consensus.cc:374] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:33:20.415480 15419 raft_consensus.cc:729] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be0827f76581485f9fc26d69b02e56ef, State: Initialized, Role: FOLLOWER
I20260808 00:33:20.415407 15417 raft_consensus.cc:348] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.415616 15419 consensus_queue.cc:260] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.415675 15417 raft_consensus.cc:374] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:33:20.415726 15417 raft_consensus.cc:729] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5608ba5c2fa4b4397156988237fd208, State: Initialized, Role: FOLLOWER
I20260808 00:33:20.415917 15419 ts_tablet_manager.cc:1428] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:33:20.415894 15417 consensus_queue.cc:260] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.415866 15422 raft_consensus.cc:348] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.416040 15391 heartbeater.cc:503] Master 127.31.210.254:45141 was elected leader, sending a full tablet report...
I20260808 00:33:20.416062 15422 raft_consensus.cc:374] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:33:20.416105 15422 raft_consensus.cc:729] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d66e2de3fdac4379a6e1c8a439e4d182, State: Initialized, Role: FOLLOWER
I20260808 00:33:20.416224 15422 consensus_queue.cc:260] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.416548 15422 ts_tablet_manager.cc:1428] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:33:20.416536 15417 ts_tablet_manager.cc:1428] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:33:20.416662 15009 heartbeater.cc:503] Master 127.31.210.254:45141 was elected leader, sending a full tablet report...
I20260808 00:33:20.416662 15194 heartbeater.cc:503] Master 127.31.210.254:45141 was elected leader, sending a full tablet report...
I20260808 00:33:20.444005 15428 raft_consensus.cc:482] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:33:20.444144 15428 raft_consensus.cc:504] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.444540 15428 leader_election.cc:288] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5608ba5c2fa4b4397156988237fd208 (127.31.210.193:41049), d66e2de3fdac4379a6e1c8a439e4d182 (127.31.210.194:38641)
I20260808 00:33:20.447790 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.444709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:41049 (local address 127.31.210.195:53977)
0808 00:33:20.444769 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.444784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.444848 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.445044 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.445045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.445047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:33:20.445140 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.445143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.445960 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.445961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.446777 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.446780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.447537 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.447538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.447626 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.447630 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.447687 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.447734 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:33:20.447868 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.444826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:53977 (local address 127.31.210.193:41049)
0808 00:33:20.444888 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.444890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.444903 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:33:20.444934 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.444936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.444947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.445050 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:33:20.445232 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.445791 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.446875 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.447461 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.447519 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.447527 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.447738 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.447787 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:33:20.448171 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.444852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:57001 (local address 127.31.210.194:38641)
0808 00:33:20.444906 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.444908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.445160 (+ 252us) server_negotiation.cc:407] Connection header received
0808 00:33:20.445195 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.445196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.445207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.445277 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:33:20.445456 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.445990 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.446965 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.447586 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.447662 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.447671 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.448059 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.448099 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:33:20.448225 15436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.444710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:38641 (local address 127.31.210.195:57001)
0808 00:33:20.444963 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.444973 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:33:20.445147 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:33:20.445282 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:33:20.445283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:33:20.445285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:33:20.445358 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.445359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.446078 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.446079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.446854 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:33:20.446857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.447742 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.447743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:33:20.447836 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.447841 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:33:20.448044 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 00:33:20.448119 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":59,"thread_start_us":116,"threads_started":1}
I20260808 00:33:20.448390 14938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219aed784ab34f8ea770e8c63b7d1e7d" candidate_uuid: "be0827f76581485f9fc26d69b02e56ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5608ba5c2fa4b4397156988237fd208" is_pre_election: true
I20260808 00:33:20.448581 14938 raft_consensus.cc:2454] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be0827f76581485f9fc26d69b02e56ef in term 0.
I20260808 00:33:20.448657 15138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219aed784ab34f8ea770e8c63b7d1e7d" candidate_uuid: "be0827f76581485f9fc26d69b02e56ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" is_pre_election: true
I20260808 00:33:20.448829 15138 raft_consensus.cc:2454] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be0827f76581485f9fc26d69b02e56ef in term 0.
I20260808 00:33:20.448829 15232 leader_election.cc:302] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be0827f76581485f9fc26d69b02e56ef, f5608ba5c2fa4b4397156988237fd208; no voters:
I20260808 00:33:20.448978 15428 raft_consensus.cc:2781] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:33:20.449059 15428 raft_consensus.cc:482] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:33:20.449086 15428 raft_consensus.cc:3037] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:33:20.449824 15428 raft_consensus.cc:504] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.449977 15428 leader_election.cc:288] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [CANDIDATE]: Term 1 election: Requested vote from peers f5608ba5c2fa4b4397156988237fd208 (127.31.210.193:41049), d66e2de3fdac4379a6e1c8a439e4d182 (127.31.210.194:38641)
I20260808 00:33:20.450150 14938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219aed784ab34f8ea770e8c63b7d1e7d" candidate_uuid: "be0827f76581485f9fc26d69b02e56ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5608ba5c2fa4b4397156988237fd208"
I20260808 00:33:20.450187 15138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219aed784ab34f8ea770e8c63b7d1e7d" candidate_uuid: "be0827f76581485f9fc26d69b02e56ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d66e2de3fdac4379a6e1c8a439e4d182"
I20260808 00:33:20.450233 14938 raft_consensus.cc:3037] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:33:20.450246 15138 raft_consensus.cc:3037] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:33:20.451180 15138 raft_consensus.cc:2454] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be0827f76581485f9fc26d69b02e56ef in term 1.
I20260808 00:33:20.451180 14938 raft_consensus.cc:2454] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be0827f76581485f9fc26d69b02e56ef in term 1.
I20260808 00:33:20.451411 15228 leader_election.cc:302] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be0827f76581485f9fc26d69b02e56ef, d66e2de3fdac4379a6e1c8a439e4d182; no voters:
I20260808 00:33:20.451541 15428 raft_consensus.cc:2781] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:33:20.451694 15428 raft_consensus.cc:686] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 1 LEADER]: Becoming Leader. State: Replica: be0827f76581485f9fc26d69b02e56ef, State: Running, Role: LEADER
I20260808 00:33:20.451797 15428 consensus_queue.cc:237] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } }
I20260808 00:33:20.452706 14740 catalog_manager.cc:5495] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef reported cstate change: term changed from 0 to 1, leader changed from <none> to be0827f76581485f9fc26d69b02e56ef (127.31.210.195). New cstate: current_term: 1 leader_uuid: "be0827f76581485f9fc26d69b02e56ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be0827f76581485f9fc26d69b02e56ef" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 35443 } health_report { overall_health: HEALTHY } } }
W20260808 00:33:20.461855 15196 tablet.cc:2369] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:33:20.464628 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.461685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.31.210.195:35443)
0808 00:33:20.461736 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:33:20.461738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:33:20.461785 (+ 47us) server_negotiation.cc:407] Connection header received
0808 00:33:20.461955 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:33:20.461956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:33:20.461985 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:33:20.462079 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:33:20.462262 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.462790 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.463657 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:33:20.463873 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:33:20.463968 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:33:20.464177 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:33:20.464299 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:33:20.464406 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:33:20.464452 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 00:33:20.464476 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:33:20.466578 15138 raft_consensus.cc:1261] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 1 FOLLOWER]: Refusing update from remote peer be0827f76581485f9fc26d69b02e56ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:33:20.466580 14938 raft_consensus.cc:1261] T 219aed784ab34f8ea770e8c63b7d1e7d P f5608ba5c2fa4b4397156988237fd208 [term 1 FOLLOWER]: Refusing update from remote peer be0827f76581485f9fc26d69b02e56ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:33:20.467010 15437 consensus_queue.cc:1035] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5608ba5c2fa4b4397156988237fd208" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:33:20.467147 15428 consensus_queue.cc:1035] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d66e2de3fdac4379a6e1c8a439e4d182" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:33:20.470852 15444 mvcc.cc:205] Tried to move back new op lower bound from 7316067125108150272 to 7316067125051207680. Current Snapshot: MvccSnapshot[applied={T|T < 7316067125108150272 or (T in {7316067125108150272})}]
I20260808 00:33:20.473186 14737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:33:20.473296 14737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:33:20.474185 14737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e3204b1184a4305bec64dea0290f6c8: Sending DeleteTablet for 3 replicas of tablet 219aed784ab34f8ea770e8c63b7d1e7d
I20260808 00:33:20.474547 15110 tablet_service.cc:1514] Processing DeleteTablet for tablet 219aed784ab34f8ea770e8c63b7d1e7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:33:20 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:56778
I20260808 00:33:20.474629 15291 tablet_service.cc:1514] Processing DeleteTablet for tablet 219aed784ab34f8ea770e8c63b7d1e7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:33:20 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:55784
I20260808 00:33:20.474840 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 14824
I20260808 00:33:20.474864 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 219aed784ab34f8ea770e8c63b7d1e7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:33:20 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:39958
I20260808 00:33:20.475504 15457 tablet_replica.cc:330] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: stopping tablet replica
I20260808 00:33:20.475622 15457 raft_consensus.cc:2229] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:33:20.475744 15457 raft_consensus.cc:2258] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:33:20.475936 15456 tablet_replica.cc:330] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: stopping tablet replica
I20260808 00:33:20.476073 15456 raft_consensus.cc:2229] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:33:20.476207 15456 raft_consensus.cc:2258] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:33:20.477711 15456 ts_tablet_manager.cc:1905] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:33:20.477842 15457 ts_tablet_manager.cc:1905] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:33:20.479761 15457 ts_tablet_manager.cc:1918] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:33:20.479761 15456 ts_tablet_manager.cc:1918] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:33:20.479874 15457 log.cc:1196] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/219aed784ab34f8ea770e8c63b7d1e7d
I20260808 00:33:20.479890 15456 log.cc:1196] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/219aed784ab34f8ea770e8c63b7d1e7d
I20260808 00:33:20.480268 15457 ts_tablet_manager.cc:1939] T 219aed784ab34f8ea770e8c63b7d1e7d P be0827f76581485f9fc26d69b02e56ef: Deleting consensus metadata
I20260808 00:33:20.480268 15456 ts_tablet_manager.cc:1939] T 219aed784ab34f8ea770e8c63b7d1e7d P d66e2de3fdac4379a6e1c8a439e4d182: Deleting consensus metadata
I20260808 00:33:20.480729 14722 catalog_manager.cc:4841] TS be0827f76581485f9fc26d69b02e56ef (127.31.210.195:35443): tablet 219aed784ab34f8ea770e8c63b7d1e7d (table test-table [id=a9d6b5aecf4c47ec9f13d672c19b7497]) successfully deleted
I20260808 00:33:20.480789 14719 catalog_manager.cc:4841] TS d66e2de3fdac4379a6e1c8a439e4d182 (127.31.210.194:38641): tablet 219aed784ab34f8ea770e8c63b7d1e7d (table test-table [id=a9d6b5aecf4c47ec9f13d672c19b7497]) successfully deleted
W20260808 00:33:20.489013 14722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:41049 (error 108)
I20260808 00:33:20.489308 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 15017
I20260808 00:33:20.490026 15411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:33:20.489502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:41049 (local address 127.0.0.1:39968)
0808 00:33:20.489927 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 00:33:20.489989 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398}
W20260808 00:33:20.490303 14722 catalog_manager.cc:4584] TS f5608ba5c2fa4b4397156988237fd208 (127.31.210.193:41049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 219aed784ab34f8ea770e8c63b7d1e7d: Network error: Client connection negotiation failed: client connection to 127.31.210.193:41049: connect: Connection refused (error 111)
I20260808 00:33:20.499184 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 15201
I20260808 00:33:20.508026 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 14694
2026-08-08T00:33:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2157 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[15462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[15462](info): set up 2 sockets
Aug 08 00:33:20 dist-test-slave-44d2 krb5kdc[15462](info): commencing operation
krb5kdc: starting...
W20260808 00:33:21.742755 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:33:21 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:33:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:33:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/rangeradmin_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/rangeradmin_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/rangerlookup_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/rangerlookup_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
I20260808 00:33:21.932025 32587 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdt1wd3/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/postgres -l logfile start
2026-08-08 00:33:24.692 UTC [15510] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 00:33:24.692 UTC [15510] LOG: listening on IPv4 address "127.31.210.212", port 39455
2026-08-08 00:33:24.696 UTC [15510] LOG: listening on Unix socket "/tmp/.s.PGSQL.39455"
2026-08-08 00:33:24.733 UTC [15514] LOG: database system was shut down at 2026-08-08 00:33:23 UTC
2026-08-08 00:33:24.738 UTC [15510] LOG: database system is ready to accept connections
I20260808 00:33:25.867733 32587 mini_postgres.cc:96] Postgres bound to 39455
2026-08-08 00:33:25.876 UTC [15527] FATAL: database "slave" does not exist
127.31.210.212:39455 - accepting connections
I20260808 00:33:25.877099 32587 mini_ranger.cc:161] Starting Ranger...
I20260808 00:33:25.893822 32587 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 00:33:26.444965 32587 mini_ranger.cc:87] Created ranger Postgres database
I20260808 00:33:26.445070 32587 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-admin
2026-08-08 00:33:26,789 [I] DB FLAVOR :POSTGRES
2026-08-08 00:33:26,789 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:33:26,790 [I] Checking connection..
2026-08-08 00:33:26,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 00:33:27,074 [I] Checking connection passed.
2026-08-08 00:33:27,074 [I] --------- Verifying version history table ---------
2026-08-08 00:33:27,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:33:27,352 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:33:27,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:33:27,621 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:33:27,621 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 00:33:27,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 00:33:27.888 UTC [15612] WARNING: there is no transaction in progress
2026-08-08 00:33:27,898 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 00:33:27,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:33:28,170 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 00:33:28,170 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:33:28,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 00:33:28,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 00:33:28,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-44d2.c.gcp-upstream.internal','N') ;"
2026-08-08 00:33:28,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:33:29,204 [I] Table x_portal_user does not exist in database ranger
2026-08-08 00:33:29,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:33:29,475 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 00:33:29,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:33:29,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:33:30,016 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 00:33:30,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 00:33:31.047 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.064 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.081 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.097 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.116 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.137 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.193 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.200 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.215 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.225 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.236 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.252 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.263 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.270 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.277 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31.284 UTC [15765] WARNING: there is no transaction in progress
2026-08-08 00:33:31,956 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 00:33:31,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 00:33:32,224 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 00:33:32,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:33:32,520 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 00:33:32,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:33:32,790 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 00:33:32,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:33:33,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:33:33,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-44d2.c.gcp-upstream.internal';"
2026-08-08 00:33:33,624 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 00:33:33,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 00:33:33,876 [I] DB_PATCHES have already been applied
I20260808 00:33:33.882543 32587 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdt1wd3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 00:33:33.882616 32587 mini_ranger.cc:193] Using host: 127.31.210.212
I20260808 00:33:33.886559 32587 mini_ranger.cc:227] Ranger admin URL: http://127.31.210.212:35365
Aug 08, 2026 12:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 12:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 12:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 12:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 12:33:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 00:33:35.265632 32587 mini_ranger.cc:160] Time spent starting Ranger: real 9.389s user 0.002s sys 0.010s
Aug 08, 2026 12:33:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 12:33:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 12:33:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 12:33:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 12:33:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 00:33:59.341398 32587 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/rangerkms_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/rangerkms_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.210.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.210.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 00:33:59 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149239, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 00:33:59.434288 32587 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 00:33:59.444893 32587 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 00:34:00.386286 32587 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 00:34:00.386396 32587 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms
I20260808 00:34:00.386423 32587 mini_ranger_kms.cc:227] Using postgres at 127.31.210.212:39455
2026-08-08 00:34:00,816 [I] DB FLAVOR :POSTGRES
2026-08-08 00:34:00,816 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:34:00,817 [I] Checking connection
2026-08-08 00:34:00,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 00:34:01,077 [I] connection success
2026-08-08 00:34:01,077 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 00:34:01,077 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 00:34:01,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 00:34:01,323 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 00:34:01,323 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:34:01,324 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 00:34:01,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:39455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 00:34:01,602 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 00:34:02.111932 32587 mini_ranger_kms.cc:310] Created kms service
I20260808 00:34:02.316334 32587 mini_ranger_kms.cc:326] Created kudu user
I20260808 00:34:02.362846 32587 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 00:34:02 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149239, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:34:02.793972 32587 mini_ranger_kms.cc:384] Added ranger policy
I20260808 00:34:02.794129 32587 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 00:34:02.794154 32587 mini_ranger_kms.cc:240] Using host: 127.31.210.212
I20260808 00:34:02.799232 32587 mini_ranger_kms.cc:276] Ranger KMS PID: 16031
I20260808 00:34:02.799374 32587 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.210.212:33547
Aug 08, 2026 12:34:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:34:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:34:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 12:34:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 12:34:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 12:34:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 00:34:05 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.210.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:05.571872 32587 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.138s user 0.006s sys 0.269s
I20260808 00:34:05.572002 32587 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 00:34:05.572094 32587 mini_ranger_kms.cc:401] 127.31.210.212:33547/kms/v1/keys
Aug 08 00:34:05 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.210.212@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
Aug 08 00:34:06 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.210.212@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
Aug 08 00:34:08 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:34:08.563647 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43649
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:43649
--ranger_config_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:33547/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:34:08.667660 16104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:34:08.667898 16104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:34:08.667937 16104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:34:08.669173 16104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:34:08.669242 16104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:34:08.669268 16104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:34:08.669301 16104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:34:08.669332 16104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:34:08.671046 16104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.210.212:33547/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:43649
--ranger_config_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:34:08.671326 16104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:34:08.671561 16104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:34:08.677383 16114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:08.677474 16110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:34:08.677425 16104 server_base.cc:1034] running on GCE node
W20260808 00:34:08.677404 16112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:34:08.677969 16104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:34:08.678328 16104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:34:08.679541 16104 hybrid_clock.cc:648] HybridClock initialized: now 1786149248679515 us; error 43 us; skew 500 ppm
Aug 08 00:34:08 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149248, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:08.682504 16104 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:34:08.683032 16104 webserver.cc:460] Webserver started at http://127.31.210.254:33105/ using document root <none> and password file <none>
I20260808 00:34:08.683291 16104 fs_manager.cc:359] Metadata directory not provided
I20260808 00:34:08.683334 16104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:34:08.683493 16104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:34:08 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149248, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:34:08.743933 16104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "ec204754539d4e8b9b4b51232896daf2"
format_stamp: "Formatted at 2026-08-08 00:34:08 on dist-test-slave-44d2"
server_key: "eca6ccda72dcd0643f16c2e814b242d4"
server_key_iv: "1a2d56752b16722c2849ceac8aa0bbfa"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:08.744571 16104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "ec204754539d4e8b9b4b51232896daf2"
format_stamp: "Formatted at 2026-08-08 00:34:08 on dist-test-slave-44d2"
server_key: "eca6ccda72dcd0643f16c2e814b242d4"
server_key_iv: "1a2d56752b16722c2849ceac8aa0bbfa"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:08.747087 16104 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:34:08.765838 16122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:08.766103 16104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:34:08.766215 16104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "ec204754539d4e8b9b4b51232896daf2"
format_stamp: "Formatted at 2026-08-08 00:34:08 on dist-test-slave-44d2"
server_key: "eca6ccda72dcd0643f16c2e814b242d4"
server_key_iv: "1a2d56752b16722c2849ceac8aa0bbfa"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:08.766345 16104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:34:08.779558 16104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:34:08.781891 16104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:34:08.782076 16104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:34:08.787516 16104 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:43649
I20260808 00:34:08.787540 16192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:43649 every 8 connection(s)
I20260808 00:34:08.788007 16104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:34:08.788836 16193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:34:08.791213 16193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2: Bootstrap starting.
I20260808 00:34:08.791774 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 16104
I20260808 00:34:08.791878 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:34:08.791929 16193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2: Neither blocks nor log segments found. Creating new log.
I20260808 00:34:08.792088 32587 external_mini_cluster.cc:1428] Setting key c68ce6f058f6fa4e153ce8c23e9868fe
I20260808 00:34:08.792201 16193 log.cc:824] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2: Log is configured to *not* fsync() on all Append() calls
I20260808 00:34:08.793128 16193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2: No bootstrap required, opened a new log
I20260808 00:34:08.795694 16193 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec204754539d4e8b9b4b51232896daf2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43649 } }
I20260808 00:34:08.795861 16193 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:34:08.795928 16193 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec204754539d4e8b9b4b51232896daf2, State: Initialized, Role: FOLLOWER
I20260808 00:34:08.796097 16193 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec204754539d4e8b9b4b51232896daf2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43649 } }
I20260808 00:34:08.796212 16193 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:34:08.796280 16193 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:34:08.796343 16193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:34:08.797076 16193 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec204754539d4e8b9b4b51232896daf2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43649 } }
I20260808 00:34:08.797196 16193 leader_election.cc:302] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec204754539d4e8b9b4b51232896daf2; no voters:
I20260808 00:34:08.797437 16193 leader_election.cc:288] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:34:08.797504 16200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:34:08.797777 16193 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:34:08.797760 16200 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [term 1 LEADER]: Becoming Leader. State: Replica: ec204754539d4e8b9b4b51232896daf2, State: Running, Role: LEADER
I20260808 00:34:08.799283 16200 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec204754539d4e8b9b4b51232896daf2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43649 } }
I20260808 00:34:08.800004 16201 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec204754539d4e8b9b4b51232896daf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec204754539d4e8b9b4b51232896daf2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43649 } } }
I20260808 00:34:08.800087 16201 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [sys.catalog]: This master's current role is: LEADER
I20260808 00:34:08.800221 16193 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:34:08.800232 16201 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec204754539d4e8b9b4b51232896daf2. Latest consensus state: current_term: 1 leader_uuid: "ec204754539d4e8b9b4b51232896daf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec204754539d4e8b9b4b51232896daf2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43649 } } }
I20260808 00:34:08.800426 16201 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2 [sys.catalog]: This master's current role is: LEADER
Aug 08 00:34:08 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:34:08.801501 16227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:34:08.802067 16227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:34:08.809190 16227 catalog_manager.cc:1347] Generated new cluster ID: 8345aadf8b7f4b01b5c9add6dd4ed126
I20260808 00:34:08.809193 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:08.792670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.31.210.254:43649)
0808 00:34:08.792855 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:08.792865 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:08.792908 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:34:08.793486 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:08.793496 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:08.793592 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:08.793761 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:34:08.794621 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:08.796076 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:08.797252 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:08.797880 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:08.798011 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:08.801177 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:34:08.801182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:34:08.801183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:34:08.801195 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:34:08.803397 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:08.803804 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:08.803807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:08.803807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:08.803922 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:08.804138 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:08.804138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:08.804139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:08.804434 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:34:08.807310 (+ 2876us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:08.807740 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 00:34:08.808667 (+ 927us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260808 00:34:08.809369 16227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:34:08.825522 16227 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:34:08.826393 16227 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:34:08.830180 16227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2: Generated new TSK 0
I20260808 00:34:08.830518 16227 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:34:09.655661 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:33547/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43649
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:34:09.784109 16265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:34:09.784430 16265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:34:09.784523 16265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:34:09.786230 16265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:34:09.786295 16265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:34:09.786338 16265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:34:09.788703 16265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.210.212:33547/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:34:09.788954 16265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:34:09.789265 16265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:34:09.795397 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:09.795428 16275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:34:09.795621 16265 server_base.cc:1034] running on GCE node
W20260808 00:34:09.795766 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:34:09.796100 16265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:34:09.796393 16265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:34:09.803819 16265 hybrid_clock.cc:648] HybridClock initialized: now 1786149249803787 us; error 42 us; skew 500 ppm
Aug 08 00:34:09 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149249, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:09.806936 16265 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:34:09.807462 16265 webserver.cc:460] Webserver started at http://127.31.210.193:44707/ using document root <none> and password file <none>
I20260808 00:34:09.807740 16265 fs_manager.cc:359] Metadata directory not provided
I20260808 00:34:09.807816 16265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:34:09.807974 16265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:34:09 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149249, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:34:09.823961 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "7263d4f26e5a4a28830403cf5dc1e7ef"
format_stamp: "Formatted at 2026-08-08 00:34:09 on dist-test-slave-44d2"
server_key: "73d2f67bf7478b44ed4ee6079d464580"
server_key_iv: "63fe1610f78b80eba042fbed8dfca1a7"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:09.824558 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "7263d4f26e5a4a28830403cf5dc1e7ef"
format_stamp: "Formatted at 2026-08-08 00:34:09 on dist-test-slave-44d2"
server_key: "73d2f67bf7478b44ed4ee6079d464580"
server_key_iv: "63fe1610f78b80eba042fbed8dfca1a7"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:09.828930 16265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260808 00:34:09.842531 16285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:09.842724 16265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:34:09.842819 16265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "7263d4f26e5a4a28830403cf5dc1e7ef"
format_stamp: "Formatted at 2026-08-08 00:34:09 on dist-test-slave-44d2"
server_key: "73d2f67bf7478b44ed4ee6079d464580"
server_key_iv: "63fe1610f78b80eba042fbed8dfca1a7"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:09.842922 16265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:34:09.867043 16265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:34:09.869765 16265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:34:09.869933 16265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:34:09.870172 16265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:34:09.870595 16265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:34:09.870656 16265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:09.870682 16265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:34:09.870694 16265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:09.877007 16265 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:38667
I20260808 00:34:09.877152 16456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:38667 every 8 connection(s)
I20260808 00:34:09.877450 16265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:34:09 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149249, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:34:09.885762 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 16265
I20260808 00:34:09.885856 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:34:09.886073 32587 external_mini_cluster.cc:1428] Setting key 59f8dc51dd6da16ec764cc2db76c6faa
I20260808 00:34:09.889715 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:09.878654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:44079 (local address 127.31.210.254:43649)
0808 00:34:09.879419 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:09.879422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:09.879431 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:34:09.879461 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:09.879462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:09.879476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:09.879566 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:34:09.880344 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:09.881207 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:09.882715 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:09.883023 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:09.883109 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:09.885996 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:34:09.885998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:34:09.885999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:34:09.886007 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:34:09.887963 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:09.888538 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:09.888538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:09.888539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:09.888607 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:09.888972 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:09.888973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:09.888974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:09.889117 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:34:09.889210 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:09.889525 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 00:34:09.889621 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":62,"threads_started":1}
I20260808 00:34:09.890113 16461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:09.878278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43649 (local address 127.31.210.193:44079)
0808 00:34:09.878495 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:09.878547 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:09.879219 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:09.879682 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:09.879687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:09.879944 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:34:09.880223 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:09.880230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:09.881321 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:09.881322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:09.882600 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:09.882603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:09.883130 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:09.883132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:09.883280 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:09.883932 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:34:09.883942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:34:09.885874 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:34:09.888069 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:34:09.888077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:34:09.888080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:34:09.888399 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:34:09.888685 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:34:09.888686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:34:09.888687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:34:09.888860 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:34:09.889198 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:34:09.889205 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:34:09.889359 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 00:34:09.889482 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 00:34:09.889646 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260808 00:34:09.890612 16457 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43649
I20260808 00:34:09.890744 16457 heartbeater.cc:464] Registering TS with master...
I20260808 00:34:09.890954 16457 heartbeater.cc:511] Master 127.31.210.254:43649 requested a full tablet report, sending...
I20260808 00:34:09.891531 16143 ts_manager.cc:194] Registered new tserver with Master: 7263d4f26e5a4a28830403cf5dc1e7ef (127.31.210.193:38667)
I20260808 00:34:09.892197 16143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:44079
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:34:09.923344 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:33547/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43649
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:34:10.035225 16470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:34:10.035501 16470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:34:10.035543 16470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:34:10.036801 16470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:34:10.036864 16470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:34:10.036908 16470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:34:10.038472 16470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.210.212:33547/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:34:10.038748 16470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:34:10.039011 16470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:34:10.044749 16489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:10.044947 16484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:10.045146 16485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:34:10.045447 16470 server_base.cc:1034] running on GCE node
I20260808 00:34:10.045778 16470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:34:10.046066 16470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:34:10.047237 16470 hybrid_clock.cc:648] HybridClock initialized: now 1786149250047195 us; error 56 us; skew 500 ppm
Aug 08 00:34:10 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149250, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:10.050643 16470 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:34:10.051203 16470 webserver.cc:460] Webserver started at http://127.31.210.194:33537/ using document root <none> and password file <none>
I20260808 00:34:10.051497 16470 fs_manager.cc:359] Metadata directory not provided
I20260808 00:34:10.051563 16470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:34:10.051744 16470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:34:10 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149250, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
Aug 08 00:34:10 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149250, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:10.065887 16470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "f66cf037c16543b283d514aa35a997eb"
format_stamp: "Formatted at 2026-08-08 00:34:10 on dist-test-slave-44d2"
server_key: "5ccc9c7a62f1d5e83e32ec7c10ec2229"
server_key_iv: "83282b4cc65c740fae88970ec235c237"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:10.066553 16470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "f66cf037c16543b283d514aa35a997eb"
format_stamp: "Formatted at 2026-08-08 00:34:10 on dist-test-slave-44d2"
server_key: "5ccc9c7a62f1d5e83e32ec7c10ec2229"
server_key_iv: "83282b4cc65c740fae88970ec235c237"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:10.068568 16470 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:34:10.081066 16498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:10.081313 16470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:34:10.081398 16470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "f66cf037c16543b283d514aa35a997eb"
format_stamp: "Formatted at 2026-08-08 00:34:10 on dist-test-slave-44d2"
server_key: "5ccc9c7a62f1d5e83e32ec7c10ec2229"
server_key_iv: "83282b4cc65c740fae88970ec235c237"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:10.081503 16470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:34:10.098913 16470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:34:10.101802 16470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:34:10.102053 16470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:34:10.102421 16470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:34:10.102980 16470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:34:10.103101 16470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:10.103197 16470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:34:10.103281 16470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:10.113924 16470 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:44173
I20260808 00:34:10.114195 16658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:44173 every 8 connection(s)
I20260808 00:34:10.114573 16470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:34:10.119210 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 16470
I20260808 00:34:10.119400 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:34:10.119644 32587 external_mini_cluster.cc:1428] Setting key 76e6b65048dbffc21418c6563ac60803
Aug 08 00:34:10 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149250, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:34:10.126521 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.115958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:57267 (local address 127.31.210.254:43649)
0808 00:34:10.116039 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.116041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.116052 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:34:10.116658 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.116660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.116682 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.116779 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:34:10.117591 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.118204 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.119714 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.120034 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.120082 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.122720 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:34:10.122722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:34:10.122722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:34:10.122732 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:34:10.124707 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:10.125237 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:10.125238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:10.125239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:10.125319 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:10.125635 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:10.125637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:10.125637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:10.125784 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:34:10.125873 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.126262 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.126395 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:34:10.126827 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.115452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43649 (local address 127.31.210.194:57267)
0808 00:34:10.115780 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.115830 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:10.116541 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:10.116850 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:10.116854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:10.117107 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:34:10.117474 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.117481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.118303 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.118304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.119608 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.119611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.120190 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.120191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.120335 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.120977 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:34:10.120990 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:34:10.122588 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:34:10.124821 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:34:10.124829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:34:10.124832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:34:10.125109 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:34:10.125416 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:34:10.125417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:34:10.125417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:34:10.125551 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:34:10.125883 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:34:10.125891 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:34:10.126088 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 00:34:10.126220 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 00:34:10.126393 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":232,"threads_started":1}
I20260808 00:34:10.127413 16659 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43649
I20260808 00:34:10.127556 16659 heartbeater.cc:464] Registering TS with master...
I20260808 00:34:10.127780 16659 heartbeater.cc:511] Master 127.31.210.254:43649 requested a full tablet report, sending...
I20260808 00:34:10.128271 16143 ts_manager.cc:194] Registered new tserver with Master: f66cf037c16543b283d514aa35a997eb (127.31.210.194:44173)
I20260808 00:34:10.128784 16143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:57267
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:34:10.156176 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:33547/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43649
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:34:10.290596 16665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:34:10.290922 16665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:34:10.290980 16665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:34:10.292779 16665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:34:10.292855 16665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:34:10.292906 16665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:34:10.295172 16665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.210.212:33547/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:34:10.295459 16665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:34:10.295775 16665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:34:10.302018 16678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:10.302026 16677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:10.302034 16681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:34:10.302237 16665 server_base.cc:1034] running on GCE node
I20260808 00:34:10.302487 16665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:34:10.302760 16665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:34:10.303925 16665 hybrid_clock.cc:648] HybridClock initialized: now 1786149250303884 us; error 58 us; skew 500 ppm
Aug 08 00:34:10 dist-test-slave-44d2 krb5kdc[15462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149250, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:10.306792 16665 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:34:10.307348 16665 webserver.cc:460] Webserver started at http://127.31.210.195:36381/ using document root <none> and password file <none>
I20260808 00:34:10.307642 16665 fs_manager.cc:359] Metadata directory not provided
I20260808 00:34:10.307715 16665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:34:10.307883 16665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:34:10 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149250, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:34:10.323921 16665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "1d182cfe68af459ebdb75c12b2389be7"
format_stamp: "Formatted at 2026-08-08 00:34:10 on dist-test-slave-44d2"
server_key: "94d892effb08c650828647f67466922a"
server_key_iv: "3f4b491055b6f84994c4684eba03ca27"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:10.326053 16665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "1d182cfe68af459ebdb75c12b2389be7"
format_stamp: "Formatted at 2026-08-08 00:34:10 on dist-test-slave-44d2"
server_key: "94d892effb08c650828647f67466922a"
server_key_iv: "3f4b491055b6f84994c4684eba03ca27"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:10.328935 16665 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 00:34:10.360675 16692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:10.372696 16665 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.001s sys 0.000s
I20260808 00:34:10.372859 16665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "1d182cfe68af459ebdb75c12b2389be7"
format_stamp: "Formatted at 2026-08-08 00:34:10 on dist-test-slave-44d2"
server_key: "94d892effb08c650828647f67466922a"
server_key_iv: "3f4b491055b6f84994c4684eba03ca27"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:10.372995 16665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:34:10.405405 16665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:34:10.407822 16665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:34:10.408069 16665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:34:10.408386 16665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:34:10.408890 16665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:34:10.408951 16665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:10.409017 16665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:34:10.409051 16665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:10.419055 16665 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:45409
I20260808 00:34:10.419575 16665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:34:10.420258 16860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:45409 every 8 connection(s)
I20260808 00:34:10.422024 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 16665
I20260808 00:34:10.422140 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:34:10.422389 32587 external_mini_cluster.cc:1428] Setting key bef2b8c5d122ec7aa8ac6ddc5e4cb800
Aug 08 00:34:10 dist-test-slave-44d2 krb5kdc[15462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149250, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:34:10.430444 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.420612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:59917 (local address 127.31.210.254:43649)
0808 00:34:10.420667 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.420669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.420936 (+ 267us) server_negotiation.cc:407] Connection header received
0808 00:34:10.421663 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.421665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.421680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.421740 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:34:10.422661 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.423403 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.424474 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.424713 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.424813 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.427184 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:34:10.427185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:34:10.427186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:34:10.427201 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:34:10.428873 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:10.429358 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:10.429360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:10.429360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:10.429430 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:10.429721 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:10.429722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:10.429723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:10.429860 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:34:10.429948 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.430249 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.430339 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:34:10.430711 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.420519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:43649 (local address 127.31.210.195:59917)
0808 00:34:10.420794 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.420829 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:10.421534 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:10.421859 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:10.421864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:10.422162 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:34:10.422525 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.422532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.423525 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.423526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.424354 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.424357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.424828 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.424829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.424979 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.425604 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:34:10.425613 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:34:10.427061 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:34:10.428981 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:34:10.428991 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:34:10.428993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:34:10.429254 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:34:10.429506 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:34:10.429507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:34:10.429508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:34:10.429644 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:34:10.429937 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:34:10.429945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:34:10.430098 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 00:34:10.430225 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 00:34:10.430401 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":87,"threads_started":1}
I20260808 00:34:10.431448 16861 heartbeater.cc:347] Connected to a master server at 127.31.210.254:43649
I20260808 00:34:10.431564 16861 heartbeater.cc:464] Registering TS with master...
I20260808 00:34:10.431773 16861 heartbeater.cc:511] Master 127.31.210.254:43649 requested a full tablet report, sending...
I20260808 00:34:10.432240 16143 ts_manager.cc:194] Registered new tserver with Master: 1d182cfe68af459ebdb75c12b2389be7 (127.31.210.195:45409)
I20260808 00:34:10.432721 16143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:59917
I20260808 00:34:10.434218 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:34:10.448661 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.436098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.31.210.254:43649)
0808 00:34:10.436141 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.436143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.436599 (+ 456us) server_negotiation.cc:407] Connection header received
0808 00:34:10.436718 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.436719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.436731 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.436764 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:34:10.437627 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.438384 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.439567 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.439869 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.439918 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.445545 (+ 5627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:34:10.445547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:34:10.445548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:34:10.445555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:34:10.447294 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:10.447731 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:10.447733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:10.447733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:10.447792 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:10.448007 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:10.448007 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:10.448008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:10.448145 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:34:10.448224 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.448460 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.448570 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:34:10.451010 16143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:34:10.453547 16143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:34:10.463785 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.457590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38667 (local address 127.0.0.1:33650)
0808 00:34:10.458422 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.458462 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:10.458531 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:10.459125 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:10.459126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:10.459129 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:34:10.459179 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.459181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.460956 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.460957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.462267 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.462270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.463475 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.463476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.463560 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.463578 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:34:10.463630 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:34:10.463683 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":120,"threads_started":1}
I20260808 00:34:10.464165 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.457850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45409 (local address 127.0.0.1:54382)
0808 00:34:10.458763 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.458770 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:10.458803 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:10.459545 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:10.459546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:10.459548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:34:10.459600 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.459602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.461557 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.461558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.462794 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.462798 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.463937 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.463938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.464004 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.464008 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:34:10.464059 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:34:10.464099 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":79,"threads_started":1}
I20260808 00:34:10.464183 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.457908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.31.210.193:38667)
0808 00:34:10.458896 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.458901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.458911 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:34:10.458944 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.458947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.458963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.459048 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:34:10.459690 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.460845 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.462372 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.463364 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.463970 (+ 606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.464045 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.464074 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.464116 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":957,"thread_start_us":64,"threads_started":1}
I20260808 00:34:10.464314 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.457663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:44173 (local address 127.0.0.1:35810)
0808 00:34:10.458629 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.458638 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:10.458688 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:10.459008 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:10.459012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:10.459015 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:34:10.459086 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.459088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.460801 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.460802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.462228 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.462231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.463688 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.463689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.463756 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.463761 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:34:10.463817 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:34:10.463861 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":33,"thread_start_us":96,"threads_started":1}
I20260808 00:34:10.464596 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.458008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35810 (local address 127.31.210.194:44173)
0808 00:34:10.458355 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.458360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.458721 (+ 361us) server_negotiation.cc:407] Connection header received
0808 00:34:10.458777 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.458779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.458799 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.458894 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:34:10.459177 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.460700 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.462673 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.463491 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.463582 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.463648 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.464438 (+ 790us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.464510 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":94,"threads_started":1}
I20260808 00:34:10.464753 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.458118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54382 (local address 127.31.210.195:45409)
0808 00:34:10.459317 (+ 1199us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.459321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.459330 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:34:10.459367 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.459369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.459384 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.459454 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:34:10.460212 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.461402 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.462956 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.463755 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.463840 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.463899 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.464621 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.464670 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1161,"thread_start_us":87,"threads_started":1}
I20260808 00:34:10.465317 16362 tablet_service.cc:1467] Processing CreateTablet for tablet a6cf74c0eb9d4de48f34afa175d388e4 (DEFAULT_TABLE table=test-table [id=4bfbd9aba1de4d248e349d3cb10240b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:34:10.465514 16566 tablet_service.cc:1467] Processing CreateTablet for tablet a6cf74c0eb9d4de48f34afa175d388e4 (DEFAULT_TABLE table=test-table [id=4bfbd9aba1de4d248e349d3cb10240b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:34:10.465592 16753 tablet_service.cc:1467] Processing CreateTablet for tablet a6cf74c0eb9d4de48f34afa175d388e4 (DEFAULT_TABLE table=test-table [id=4bfbd9aba1de4d248e349d3cb10240b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:34:10.465742 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6cf74c0eb9d4de48f34afa175d388e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:34:10.465816 16566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6cf74c0eb9d4de48f34afa175d388e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:34:10.465946 16753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6cf74c0eb9d4de48f34afa175d388e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:34:10.469676 16888 tablet_bootstrap.cc:492] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef: Bootstrap starting.
I20260808 00:34:10.470186 16889 tablet_bootstrap.cc:492] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Bootstrap starting.
I20260808 00:34:10.470371 16890 tablet_bootstrap.cc:492] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Bootstrap starting.
I20260808 00:34:10.470541 16888 tablet_bootstrap.cc:654] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef: Neither blocks nor log segments found. Creating new log.
I20260808 00:34:10.471010 16888 log.cc:824] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef: Log is configured to *not* fsync() on all Append() calls
I20260808 00:34:10.471280 16889 tablet_bootstrap.cc:654] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Neither blocks nor log segments found. Creating new log.
I20260808 00:34:10.471628 16890 tablet_bootstrap.cc:654] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Neither blocks nor log segments found. Creating new log.
I20260808 00:34:10.471693 16889 log.cc:824] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:34:10.471907 16888 tablet_bootstrap.cc:492] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef: No bootstrap required, opened a new log
I20260808 00:34:10.471989 16888 ts_tablet_manager.cc:1397] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:34:10.472038 16890 log.cc:824] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Log is configured to *not* fsync() on all Append() calls
I20260808 00:34:10.473481 16890 tablet_bootstrap.cc:492] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: No bootstrap required, opened a new log
I20260808 00:34:10.473891 16889 tablet_bootstrap.cc:492] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: No bootstrap required, opened a new log
I20260808 00:34:10.473960 16889 ts_tablet_manager.cc:1397] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 00:34:10.474689 16888 raft_consensus.cc:348] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.474879 16888 raft_consensus.cc:374] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:34:10.474918 16888 raft_consensus.cc:729] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7263d4f26e5a4a28830403cf5dc1e7ef, State: Initialized, Role: FOLLOWER
I20260808 00:34:10.475040 16888 consensus_queue.cc:260] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.475304 16888 ts_tablet_manager.cc:1428] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:34:10.475350 16457 heartbeater.cc:503] Master 127.31.210.254:43649 was elected leader, sending a full tablet report...
I20260808 00:34:10.475579 16890 ts_tablet_manager.cc:1397] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 00:34:10.476087 16889 raft_consensus.cc:348] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.476250 16889 raft_consensus.cc:374] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:34:10.476284 16889 raft_consensus.cc:729] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f66cf037c16543b283d514aa35a997eb, State: Initialized, Role: FOLLOWER
I20260808 00:34:10.476426 16889 consensus_queue.cc:260] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.476737 16889 ts_tablet_manager.cc:1428] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 00:34:10.476804 16659 heartbeater.cc:503] Master 127.31.210.254:43649 was elected leader, sending a full tablet report...
I20260808 00:34:10.478061 16890 raft_consensus.cc:348] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.478252 16890 raft_consensus.cc:374] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:34:10.478312 16890 raft_consensus.cc:729] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d182cfe68af459ebdb75c12b2389be7, State: Initialized, Role: FOLLOWER
I20260808 00:34:10.478438 16890 consensus_queue.cc:260] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.478770 16890 ts_tablet_manager.cc:1428] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:34:10.478979 16861 heartbeater.cc:503] Master 127.31.210.254:43649 was elected leader, sending a full tablet report...
I20260808 00:34:10.523574 16897 raft_consensus.cc:482] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:34:10.523754 16897 raft_consensus.cc:504] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.524189 16897 leader_election.cc:288] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7263d4f26e5a4a28830403cf5dc1e7ef (127.31.210.193:38667), 1d182cfe68af459ebdb75c12b2389be7 (127.31.210.195:45409)
I20260808 00:34:10.528267 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.524371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38667 (local address 127.31.210.194:35165)
0808 00:34:10.524520 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.524533 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:10.524584 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:10.524861 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:10.524862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:10.524865 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:34:10.524953 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.524956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.525858 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.525860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.527045 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.527048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.527989 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.527990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.528070 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.528074 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:34:10.528127 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:34:10.528170 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260808 00:34:10.528518 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.524747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:36631 (local address 127.31.210.195:45409)
0808 00:34:10.524793 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.524795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.525186 (+ 391us) server_negotiation.cc:407] Connection header received
0808 00:34:10.525318 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.525319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.525330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.525358 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:34:10.525585 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.526296 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.527603 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.528116 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.528161 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.528168 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.528378 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.528426 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:34:10.528823 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.524639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:35165 (local address 127.31.210.193:38667)
0808 00:34:10.524698 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.524700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.524709 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:34:10.524740 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.524741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.524751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.524801 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:34:10.525035 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.525750 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.527234 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.527846 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.527940 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.527952 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.528706 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.528756 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:34:10.528931 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.524822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45409 (local address 127.31.210.194:36631)
0808 00:34:10.525115 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.525125 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:34:10.525214 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:34:10.525434 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:34:10.525434 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:34:10.525436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:34:10.525514 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.525515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.526430 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.526430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.527504 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:34:10.527506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.528227 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.528227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:34:10.528300 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.528303 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:34:10.528793 (+ 490us) client_negotiation.cc:240] Negotiation successful
0808 00:34:10.528839 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":222,"threads_started":1}
I20260808 00:34:10.529127 16391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cf74c0eb9d4de48f34afa175d388e4" candidate_uuid: "f66cf037c16543b283d514aa35a997eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" is_pre_election: true
I20260808 00:34:10.529290 16391 raft_consensus.cc:2454] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f66cf037c16543b283d514aa35a997eb in term 0.
I20260808 00:34:10.529256 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cf74c0eb9d4de48f34afa175d388e4" candidate_uuid: "f66cf037c16543b283d514aa35a997eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d182cfe68af459ebdb75c12b2389be7" is_pre_election: true
I20260808 00:34:10.529458 16791 raft_consensus.cc:2454] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f66cf037c16543b283d514aa35a997eb in term 0.
I20260808 00:34:10.529572 16501 leader_election.cc:302] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7263d4f26e5a4a28830403cf5dc1e7ef, f66cf037c16543b283d514aa35a997eb; no voters:
I20260808 00:34:10.529771 16897 raft_consensus.cc:2781] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:34:10.529834 16897 raft_consensus.cc:482] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:34:10.529860 16897 raft_consensus.cc:3037] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:34:10.530823 16897 raft_consensus.cc:504] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.530979 16897 leader_election.cc:288] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [CANDIDATE]: Term 1 election: Requested vote from peers 7263d4f26e5a4a28830403cf5dc1e7ef (127.31.210.193:38667), 1d182cfe68af459ebdb75c12b2389be7 (127.31.210.195:45409)
I20260808 00:34:10.531296 16791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cf74c0eb9d4de48f34afa175d388e4" candidate_uuid: "f66cf037c16543b283d514aa35a997eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d182cfe68af459ebdb75c12b2389be7"
I20260808 00:34:10.531387 16791 raft_consensus.cc:3037] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:34:10.532243 16391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6cf74c0eb9d4de48f34afa175d388e4" candidate_uuid: "f66cf037c16543b283d514aa35a997eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef"
I20260808 00:34:10.532330 16791 raft_consensus.cc:2454] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f66cf037c16543b283d514aa35a997eb in term 1.
I20260808 00:34:10.532325 16391 raft_consensus.cc:3037] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:34:10.532552 16504 leader_election.cc:302] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d182cfe68af459ebdb75c12b2389be7, f66cf037c16543b283d514aa35a997eb; no voters:
I20260808 00:34:10.532660 16897 raft_consensus.cc:2781] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:34:10.532902 16897 raft_consensus.cc:686] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 1 LEADER]: Becoming Leader. State: Replica: f66cf037c16543b283d514aa35a997eb, State: Running, Role: LEADER
I20260808 00:34:10.533003 16897 consensus_queue.cc:237] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } }
I20260808 00:34:10.533336 16391 raft_consensus.cc:2454] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f66cf037c16543b283d514aa35a997eb in term 1.
I20260808 00:34:10.533860 16143 catalog_manager.cc:5495] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb reported cstate change: term changed from 0 to 1, leader changed from <none> to f66cf037c16543b283d514aa35a997eb (127.31.210.194). New cstate: current_term: 1 leader_uuid: "f66cf037c16543b283d514aa35a997eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f66cf037c16543b283d514aa35a997eb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 } health_report { overall_health: UNKNOWN } } }
I20260808 00:34:10.540575 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.537416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.31.210.194:44173)
0808 00:34:10.537457 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:10.537459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:10.537466 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:34:10.537500 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:10.537501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:10.537521 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:10.537589 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:34:10.537789 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.538519 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.539574 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:10.539881 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:10.539954 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:10.540187 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:34:10.540317 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:34:10.540417 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:10.540460 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 00:34:10.540504 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:34:10.542733 16791 raft_consensus.cc:1261] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 1 FOLLOWER]: Refusing update from remote peer f66cf037c16543b283d514aa35a997eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:34:10.542776 16391 raft_consensus.cc:1261] T a6cf74c0eb9d4de48f34afa175d388e4 P 7263d4f26e5a4a28830403cf5dc1e7ef [term 1 FOLLOWER]: Refusing update from remote peer f66cf037c16543b283d514aa35a997eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:34:10.543123 16902 consensus_queue.cc:1035] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7263d4f26e5a4a28830403cf5dc1e7ef" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:34:10.543231 16897 consensus_queue.cc:1035] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d182cfe68af459ebdb75c12b2389be7" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:34:10.547391 16905 mvcc.cc:205] Tried to move back new op lower bound from 7316067330220048384 to 7316067330183614464. Current Snapshot: MvccSnapshot[applied={T|T < 7316067330220048384}]
I20260808 00:34:10.547690 16907 mvcc.cc:205] Tried to move back new op lower bound from 7316067330220048384 to 7316067330183614464. Current Snapshot: MvccSnapshot[applied={T|T < 7316067330220048384}]
I20260808 00:34:10.551163 16908 mvcc.cc:205] Tried to move back new op lower bound from 7316067330220048384 to 7316067330183614464. Current Snapshot: MvccSnapshot[applied={T|T < 7316067330220048384}]
I20260808 00:34:10.551368 16142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:34:10.551474 16142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:34:10.552386 16142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec204754539d4e8b9b4b51232896daf2: Sending DeleteTablet for 3 replicas of tablet a6cf74c0eb9d4de48f34afa175d388e4
I20260808 00:34:10.552999 16753 tablet_service.cc:1514] Processing DeleteTablet for tablet a6cf74c0eb9d4de48f34afa175d388e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:34:10 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:54382
I20260808 00:34:10.553686 16362 tablet_service.cc:1514] Processing DeleteTablet for tablet a6cf74c0eb9d4de48f34afa175d388e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:34:10 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:33650
I20260808 00:34:10.554414 16566 tablet_service.cc:1514] Processing DeleteTablet for tablet a6cf74c0eb9d4de48f34afa175d388e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:34:10 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:35810
I20260808 00:34:10.555013 16753 tablet_service.cc:1514] Processing DeleteTablet for tablet a6cf74c0eb9d4de48f34afa175d388e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:34:10 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:54382
I20260808 00:34:10.555282 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 16265
I20260808 00:34:10.556587 16920 tablet_replica.cc:330] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: stopping tablet replica
I20260808 00:34:10.556699 16920 raft_consensus.cc:2229] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:34:10.556809 16920 raft_consensus.cc:2258] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:34:10.561719 16920 ts_tablet_manager.cc:1905] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:34:10.564270 16920 ts_tablet_manager.cc:1918] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:34:10.564359 16920 log.cc:1196] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/a6cf74c0eb9d4de48f34afa175d388e4
I20260808 00:34:10.564621 16922 tablet_replica.cc:330] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: stopping tablet replica
I20260808 00:34:10.564741 16922 raft_consensus.cc:2229] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:34:10.564846 16922 raft_consensus.cc:2258] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:34:10.565014 16920 ts_tablet_manager.cc:1939] T a6cf74c0eb9d4de48f34afa175d388e4 P 1d182cfe68af459ebdb75c12b2389be7: Deleting consensus metadata
I20260808 00:34:10.565420 16922 ts_tablet_manager.cc:1905] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:34:10.565497 16126 catalog_manager.cc:4841] TS 1d182cfe68af459ebdb75c12b2389be7 (127.31.210.195:45409): tablet a6cf74c0eb9d4de48f34afa175d388e4 (table test-table [id=4bfbd9aba1de4d248e349d3cb10240b6]) successfully deleted
I20260808 00:34:10.567183 16922 ts_tablet_manager.cc:1918] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:34:10.567248 16922 log.cc:1196] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/a6cf74c0eb9d4de48f34afa175d388e4
I20260808 00:34:10.567523 16922 ts_tablet_manager.cc:1939] T a6cf74c0eb9d4de48f34afa175d388e4 P f66cf037c16543b283d514aa35a997eb: Deleting consensus metadata
W20260808 00:34:10.568893 16126 catalog_manager.cc:4808] TS 1d182cfe68af459ebdb75c12b2389be7 (127.31.210.195:45409): delete failed for tablet a6cf74c0eb9d4de48f34afa175d388e4 because the tablet was not found. No further retry: Not found: Tablet not found: a6cf74c0eb9d4de48f34afa175d388e4
I20260808 00:34:10.569132 16126 catalog_manager.cc:4841] TS f66cf037c16543b283d514aa35a997eb (127.31.210.194:44173): tablet a6cf74c0eb9d4de48f34afa175d388e4 (table test-table [id=4bfbd9aba1de4d248e349d3cb10240b6]) successfully deleted
W20260808 00:34:10.578768 16124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:38667 (error 108)
I20260808 00:34:10.579658 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 16470
I20260808 00:34:10.591883 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:10.591686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38667 (local address 127.0.0.1:33664)
0808 00:34:10.591760 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 00:34:10.591819 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 00:34:10.592242 16124 catalog_manager.cc:4584] TS 7263d4f26e5a4a28830403cf5dc1e7ef (127.31.210.193:38667): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6cf74c0eb9d4de48f34afa175d388e4: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38667: connect: Connection refused (error 111)
I20260808 00:34:10.600087 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 16665
I20260808 00:34:10.628425 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 16104
I20260808 00:34:10.642297 32587 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 00:34:10.988760 32587 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 00:34:10.988906 32587 mini_ranger.cc:66] Stopping Ranger...
I20260808 00:34:11.344461 32587 mini_ranger.cc:68] Stopped Ranger
2026-08-08 00:34:11.344 UTC [15510] LOG: received smart shutdown request
2026-08-08 00:34:11.349 UTC [15510] LOG: background worker "logical replication launcher" (PID 15520) exited with exit code 1
2026-08-08 00:34:11.349 UTC [15515] LOG: shutting down
2026-08-08 00:34:11.538 UTC [15510] LOG: database system is shut down
2026-08-08T00:34:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51162 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:34:11 dist-test-slave-44d2 krb5kdc[16934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:34:11 dist-test-slave-44d2 krb5kdc[16934](info): set up 2 sockets
Aug 08 00:34:11 dist-test-slave-44d2 krb5kdc[16934](info): commencing operation
krb5kdc: starting...
W20260808 00:34:12.952597 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:34:13 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:34:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/rangeradmin_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/rangeradmin_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/rangerlookup_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/rangerlookup_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
I20260808 00:34:13.147292 32587 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdt1wd3/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/postgres -l logfile start
2026-08-08 00:34:15.827 UTC [16981] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 00:34:15.827 UTC [16981] LOG: listening on IPv4 address "127.31.210.212", port 55817
2026-08-08 00:34:15.832 UTC [16981] LOG: listening on Unix socket "/tmp/.s.PGSQL.55817"
2026-08-08 00:34:15.870 UTC [16985] LOG: database system was shut down at 2026-08-08 00:34:14 UTC
2026-08-08 00:34:15.875 UTC [16981] LOG: database system is ready to accept connections
I20260808 00:34:17.041131 32587 mini_postgres.cc:96] Postgres bound to 55817
2026-08-08 00:34:17.049 UTC [16996] FATAL: database "slave" does not exist
127.31.210.212:55817 - accepting connections
I20260808 00:34:17.049820 32587 mini_ranger.cc:161] Starting Ranger...
I20260808 00:34:17.066901 32587 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 00:34:17.611573 32587 mini_ranger.cc:87] Created ranger Postgres database
I20260808 00:34:17.611675 32587 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-admin
2026-08-08 00:34:17,915 [I] DB FLAVOR :POSTGRES
2026-08-08 00:34:17,916 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:34:17,916 [I] Checking connection..
2026-08-08 00:34:17,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 00:34:18,186 [I] Checking connection passed.
2026-08-08 00:34:18,187 [I] --------- Verifying version history table ---------
2026-08-08 00:34:18,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:34:18,474 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:34:18,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:34:18,737 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:34:18,737 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 00:34:18,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 00:34:18.989 UTC [17091] WARNING: there is no transaction in progress
2026-08-08 00:34:18,998 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 00:34:18,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:34:19,278 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 00:34:19,278 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:34:19,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 00:34:19,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 00:34:19,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-44d2.c.gcp-upstream.internal','N') ;"
2026-08-08 00:34:20,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:34:20,333 [I] Table x_portal_user does not exist in database ranger
2026-08-08 00:34:20,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:34:20,602 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 00:34:20,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:34:20,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:34:21,115 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 00:34:21,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 00:34:22.165 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.183 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.201 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.218 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.237 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.257 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.323 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.330 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.346 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.356 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.367 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.381 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.391 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.398 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.404 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:22.411 UTC [17266] WARNING: there is no transaction in progress
2026-08-08 00:34:23,127 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 00:34:23,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 00:34:23,392 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 00:34:23,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:34:23,657 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 00:34:23,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:34:23,917 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 00:34:23,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:34:24,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:34:24,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-44d2.c.gcp-upstream.internal';"
2026-08-08 00:34:24,703 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 00:34:24,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 00:34:24,983 [I] DB_PATCHES have already been applied
I20260808 00:34:24.989835 32587 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdt1wd3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 00:34:24.989967 32587 mini_ranger.cc:193] Using host: 127.31.210.212
I20260808 00:34:24.994297 32587 mini_ranger.cc:227] Ranger admin URL: http://127.31.210.212:51357
Aug 08, 2026 12:34:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:34:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:34:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 12:34:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 12:34:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 12:34:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 12:34:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 00:34:26.332645 32587 mini_ranger.cc:160] Time spent starting Ranger: real 9.283s user 0.000s sys 0.011s
Aug 08, 2026 12:34:47 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 12:34:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 12:34:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 12:34:47 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 12:34:48 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 00:34:49.491729 32587 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.210.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.210.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/rangerkms_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.210.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/rangerkms_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.210.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.210.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.210.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/HTTP_127.31.210.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 00:34:49 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149289, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 00:34:49.596726 32587 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 00:34:49.608274 32587 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 00:34:50.607131 32587 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 00:34:50.607213 32587 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms
I20260808 00:34:50.607225 32587 mini_ranger_kms.cc:227] Using postgres at 127.31.210.212:55817
2026-08-08 00:34:50,795 [I] DB FLAVOR :POSTGRES
2026-08-08 00:34:50,795 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:34:50,795 [I] Checking connection
2026-08-08 00:34:50,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 00:34:51,068 [I] connection success
2026-08-08 00:34:51,069 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 00:34:51,069 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 00:34:51,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 00:34:51,341 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 00:34:51,341 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:34:51,341 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 00:34:51,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.210.212:55817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 00:34:51,632 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 00:34:52.181239 32587 mini_ranger_kms.cc:310] Created kms service
I20260808 00:34:52.370208 32587 mini_ranger_kms.cc:326] Created kudu user
I20260808 00:34:52.418983 32587 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 00:34:52 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149289, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:34:52.867663 32587 mini_ranger_kms.cc:384] Added ranger policy
I20260808 00:34:52.867794 32587 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdt1wd3/build/release/bin/postgresql.jar:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdt1wd3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 00:34:52.867815 32587 mini_ranger_kms.cc:240] Using host: 127.31.210.212
I20260808 00:34:52.872418 32587 mini_ranger_kms.cc:276] Ranger KMS PID: 17523
I20260808 00:34:52.872529 32587 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.210.212:40807
Aug 08, 2026 12:34:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:34:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 12:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 12:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 12:34:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 00:34:55 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149295, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.210.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:55.801123 32587 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.204s user 0.008s sys 0.164s
I20260808 00:34:55.801256 32587 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 00:34:55.801314 32587 mini_ranger_kms.cc:401] 127.31.210.212:40807/kms/v1/keys
Aug 08 00:34:55 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149295, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.210.212@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
Aug 08 00:34:56 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149295, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.210.212@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
Aug 08 00:34:58 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:34:58.744366 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46157
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46157
--ranger_config_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:34:58.829023 17601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:34:58.829247 17601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:34:58.829269 17601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:34:58.830542 17601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:34:58.830595 17601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:34:58.830608 17601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:34:58.830621 17601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:34:58.830633 17601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:34:58.832273 17601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.210.212:40807/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46157
--ranger_config_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:34:58.832576 17601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:34:58.832854 17601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:34:58.839430 17601 server_base.cc:1034] running on GCE node
W20260808 00:34:58.839386 17609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:58.839399 17608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:34:58.839546 17611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:34:58.839941 17601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:34:58.840252 17601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:34:58.841432 17601 hybrid_clock.cc:648] HybridClock initialized: now 1786149298841412 us; error 47 us; skew 500 ppm
Aug 08 00:34:58 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:34:58.844110 17601 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:34:58.844712 17601 webserver.cc:460] Webserver started at http://127.31.210.254:32881/ using document root <none> and password file <none>
I20260808 00:34:58.844945 17601 fs_manager.cc:359] Metadata directory not provided
I20260808 00:34:58.845011 17601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:34:58.845156 17601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:34:58 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:34:58.908433 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "ffc4317c271c45ec9b04a29a6d16d677"
format_stamp: "Formatted at 2026-08-08 00:34:58 on dist-test-slave-44d2"
server_key: "6756d4c02086a2f6417f590907b735c9"
server_key_iv: "c1a8d42bff32944e09a9fdedfedfdabe"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:58.908946 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "ffc4317c271c45ec9b04a29a6d16d677"
format_stamp: "Formatted at 2026-08-08 00:34:58 on dist-test-slave-44d2"
server_key: "6756d4c02086a2f6417f590907b735c9"
server_key_iv: "c1a8d42bff32944e09a9fdedfedfdabe"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:58.910810 17601 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:34:58.925446 17625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:34:58.925729 17601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:34:58.925848 17601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "ffc4317c271c45ec9b04a29a6d16d677"
format_stamp: "Formatted at 2026-08-08 00:34:58 on dist-test-slave-44d2"
server_key: "6756d4c02086a2f6417f590907b735c9"
server_key_iv: "c1a8d42bff32944e09a9fdedfedfdabe"
server_key_version: "kuduclusterkey@0"
I20260808 00:34:58.925976 17601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:34:58.953826 17601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:34:58.955866 17601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:34:58.956059 17601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:34:58.961342 17601 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46157
I20260808 00:34:58.961422 17696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46157 every 8 connection(s)
I20260808 00:34:58.961828 17601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:34:58.962602 17697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:34:58.964020 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 17601
I20260808 00:34:58.964138 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:34:58.964344 32587 external_mini_cluster.cc:1428] Setting key 4d7cfeea0aac88dc6b5573232d9d1fe3
I20260808 00:34:58.965709 17697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Bootstrap starting.
I20260808 00:34:58.966328 17697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Neither blocks nor log segments found. Creating new log.
I20260808 00:34:58.966647 17697 log.cc:824] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Log is configured to *not* fsync() on all Append() calls
I20260808 00:34:58.967489 17697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: No bootstrap required, opened a new log
I20260808 00:34:58.969274 17697 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:34:58.969447 17697 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:34:58.969515 17697 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc4317c271c45ec9b04a29a6d16d677, State: Initialized, Role: FOLLOWER
I20260808 00:34:58.969664 17697 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:34:58.969781 17697 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:34:58.969839 17697 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:34:58.969916 17697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:34:58.970471 17697 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:34:58.970613 17697 leader_election.cc:302] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc4317c271c45ec9b04a29a6d16d677; no voters:
I20260808 00:34:58.970876 17697 leader_election.cc:288] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:34:58.970940 17703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:34:58.971272 17703 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 LEADER]: Becoming Leader. State: Replica: ffc4317c271c45ec9b04a29a6d16d677, State: Running, Role: LEADER
I20260808 00:34:58.971310 17697 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:34:58.971468 17703 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:34:58.971872 17703 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffc4317c271c45ec9b04a29a6d16d677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } } }
I20260808 00:34:58.972012 17703 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: This master's current role is: LEADER
I20260808 00:34:58.972201 17703 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc4317c271c45ec9b04a29a6d16d677. Latest consensus state: current_term: 1 leader_uuid: "ffc4317c271c45ec9b04a29a6d16d677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } } }
I20260808 00:34:58.972307 17703 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: This master's current role is: LEADER
I20260808 00:34:58.972458 17720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:34:58.973042 17720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:34:58.973351 17697 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 08 00:34:58 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:34:58.983253 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:34:58.964974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.31.210.254:46157)
0808 00:34:58.966903 (+ 1929us) server_negotiation.cc:206] Beginning negotiation
0808 00:34:58.966910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:34:58.966937 (+ 27us) server_negotiation.cc:407] Connection header received
0808 00:34:58.967483 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:34:58.967492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:34:58.967590 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:34:58.968633 (+ 1043us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:34:58.968739 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:58.970531 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:58.971827 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:34:58.972584 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:34:58.972696 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:34:58.977244 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:34:58.977307 (+ 63us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:34:58.977308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:34:58.977427 (+ 119us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:34:58.980761 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:58.981246 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:58.981249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:58.981249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:58.981362 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:34:58.981621 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:34:58.981621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:34:58.981622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:34:58.982008 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:34:58.982663 (+ 655us) server_negotiation.cc:1035] Waiting for connection context
0808 00:34:58.982761 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 00:34:58.982956 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1887,"thread_start_us":74,"threads_started":1}
I20260808 00:34:58.985088 17720 catalog_manager.cc:1347] Generated new cluster ID: 76304353827848a09b8b9dbca1fd1648
I20260808 00:34:58.985154 17720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:34:58.995468 17720 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:34:58.996302 17720 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:34:59.002101 17720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Generated new TSK 0
I20260808 00:34:59.002379 17720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:34:59.871730 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46157
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:34:59.990572 17769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:34:59.990796 17769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:34:59.990823 17769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:34:59.991997 17769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:34:59.992048 17769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:34:59.992076 17769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:34:59.993721 17769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.210.212:40807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:34:59.994001 17769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:34:59.994282 17769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:00.001313 17777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.001389 17779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.001678 17776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:00.001883 17769 server_base.cc:1034] running on GCE node
I20260808 00:35:00.002241 17769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:00.002578 17769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:00.003836 17769 hybrid_clock.cc:648] HybridClock initialized: now 1786149300003793 us; error 53 us; skew 500 ppm
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:00.006937 17769 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:35:00.007459 17769 webserver.cc:460] Webserver started at http://127.31.210.193:33885/ using document root <none> and password file <none>
I20260808 00:35:00.007731 17769 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:00.007808 17769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:00.007970 17769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:00.024329 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "f1f024e9b9c44976a085537f8c7ec8ae"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "9d1c4bb8ee9753deda61e0259d65493a"
server_key_iv: "e667a5ee2021da31f97a92fd947b4842"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.024973 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "f1f024e9b9c44976a085537f8c7ec8ae"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "9d1c4bb8ee9753deda61e0259d65493a"
server_key_iv: "e667a5ee2021da31f97a92fd947b4842"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.028970 17769 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 00:35:00.043833 17791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.044078 17769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:00.044183 17769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "f1f024e9b9c44976a085537f8c7ec8ae"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "9d1c4bb8ee9753deda61e0259d65493a"
server_key_iv: "e667a5ee2021da31f97a92fd947b4842"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.044291 17769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:00.055540 17769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:00.058079 17769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:00.058229 17769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:00.058483 17769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:00.058847 17769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:00.058897 17769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.058928 17769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:00.058946 17769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.066552 17769 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:34731
I20260808 00:35:00.066999 17769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:35:00.068776 17963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:34731 every 8 connection(s)
I20260808 00:35:00.070102 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 17769
I20260808 00:35:00.070199 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:35:00.070421 32587 external_mini_cluster.cc:1428] Setting key b7366192c4bd79f4f04bca0fb74f6310
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:00.086074 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:00.068229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46157 (local address 127.31.210.193:52991)
0808 00:35:00.072322 (+ 4093us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:00.072379 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:00.073190 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:00.075051 (+ 1861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:00.075057 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:00.075315 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:00.075618 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:00.075625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.076652 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.076653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:00.077972 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:00.077975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.078561 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.078562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:00.078713 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:00.079338 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:00.079349 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:00.081556 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:00.084279 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:00.084288 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:00.084290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:00.084663 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:00.084905 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:00.084906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:00.084907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:00.085055 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:00.085354 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:00.085360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:00.085545 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 00:35:00.085649 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:35:00.085818 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4038,"thread_start_us":100,"threads_started":1}
I20260808 00:35:00.086546 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:00.068688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:52991 (local address 127.31.210.254:46157)
0808 00:35:00.074755 (+ 6067us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:00.074757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:00.074768 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:35:00.074798 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:00.074799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:00.074814 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:00.074921 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:00.075740 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.076542 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.078079 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.078400 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.078464 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:00.081690 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:00.081692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:00.081692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:00.081701 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:00.084175 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:00.084761 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:00.084762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:00.084762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:00.084834 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:00.085126 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:00.085127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:00.085127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:00.085269 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:00.085327 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:00.086339 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0808 00:35:00.086436 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6039,"thread_start_us":75,"threads_started":1}
I20260808 00:35:00.087036 17964 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46157
I20260808 00:35:00.087136 17964 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:00.087337 17964 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:00.087906 17647 ts_manager.cc:194] Registered new tserver with Master: f1f024e9b9c44976a085537f8c7ec8ae (127.31.210.193:34731)
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
I20260808 00:35:00.088953 17647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:52991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:35:00.116389 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46157
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:00.239326 17977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:00.239713 17977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:00.239805 17977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:00.241706 17977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:00.241842 17977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:00.241942 17977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:35:00.244238 17977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.210.212:40807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:00.244805 17977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:00.245179 17977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:00.262491 17987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.262502 17986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.263003 17991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:00.264194 17977 server_base.cc:1034] running on GCE node
I20260808 00:35:00.264456 17977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:00.264807 17977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:00.266004 17977 hybrid_clock.cc:648] HybridClock initialized: now 1786149300265972 us; error 36 us; skew 500 ppm
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:00.269793 17977 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:35:00.270414 17977 webserver.cc:460] Webserver started at http://127.31.210.194:35559/ using document root <none> and password file <none>
I20260808 00:35:00.270715 17977 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:00.270773 17977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:00.270907 17977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:00.287259 17977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "f146e2ee35f046c9afbd281b7bcff40c"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "ec4aff9c8110e2242716776c2fb6209d"
server_key_iv: "c2ed107a0824f64e5338063c64fdb2a2"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.287923 17977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "f146e2ee35f046c9afbd281b7bcff40c"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "ec4aff9c8110e2242716776c2fb6209d"
server_key_iv: "c2ed107a0824f64e5338063c64fdb2a2"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.297369 17977 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260808 00:35:00.356639 18001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.356945 17977 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260808 00:35:00.357050 17977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "f146e2ee35f046c9afbd281b7bcff40c"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "ec4aff9c8110e2242716776c2fb6209d"
server_key_iv: "c2ed107a0824f64e5338063c64fdb2a2"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.357167 17977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:00.418843 17977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:00.423410 17977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:00.425645 17977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:00.426030 17977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:00.426554 17977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:00.426669 17977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.426769 17977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:00.426841 17977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.436616 17977 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:34295
I20260808 00:35:00.436961 18170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:34295 every 8 connection(s)
I20260808 00:35:00.437379 17977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:00.447072 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 17977
I20260808 00:35:00.447184 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:35:00.447381 32587 external_mini_cluster.cc:1428] Setting key c660d5b6ab3ac80e0d3c5d46059c0ab7
I20260808 00:35:00.452548 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:00.438859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:58883 (local address 127.31.210.254:46157)
0808 00:35:00.439226 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:00.439229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:00.439244 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:35:00.439968 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:00.439970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:00.439988 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:00.440080 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:00.440942 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.441725 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.443337 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.443655 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.443730 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:00.446532 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:00.446534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:00.446535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:00.446543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:00.448949 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:00.451188 (+ 2239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:00.451190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:00.451191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:00.451265 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:00.451604 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:00.451606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:00.451606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:00.451763 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:00.451811 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:00.452281 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 00:35:00.452394 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333}
I20260808 00:35:00.452765 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:00.438533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46157 (local address 127.31.210.194:58883)
0808 00:35:00.439060 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:00.439118 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:00.439814 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:00.440193 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:00.440199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:00.440474 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:00.440832 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:00.440838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.441833 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.441834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:00.443235 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:00.443237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.443772 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.443773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:00.443951 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:00.444650 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:00.444661 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:00.446423 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:00.450656 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:00.450666 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:00.450669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:00.451075 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:00.451350 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:00.451351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:00.451351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:00.451516 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:00.451865 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:00.451872 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:00.452079 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 00:35:00.452253 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 00:35:00.452432 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":100,"threads_started":1}
I20260808 00:35:00.453433 18171 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46157
I20260808 00:35:00.453543 18171 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:00.453760 18171 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:00.454242 17647 ts_manager.cc:194] Registered new tserver with Master: f146e2ee35f046c9afbd281b7bcff40c (127.31.210.194:34295)
I20260808 00:35:00.454802 17647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:58883
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:35:00.484344 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46157
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:00.608219 18178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:00.608618 18178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:00.608711 18178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:00.610450 18178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:00.610675 18178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:00.610764 18178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:35:00.613144 18178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.210.212:40807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:00.613626 18178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:00.613951 18178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:00.620657 18185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.620882 18186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.621024 18189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:00.622124 18178 server_base.cc:1034] running on GCE node
I20260808 00:35:00.622344 18178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:00.622608 18178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:00.623772 18178 hybrid_clock.cc:648] HybridClock initialized: now 1786149300623750 us; error 34 us; skew 500 ppm
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:00.627198 18178 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:35:00.627707 18178 webserver.cc:460] Webserver started at http://127.31.210.195:35707/ using document root <none> and password file <none>
I20260808 00:35:00.628008 18178 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:00.628077 18178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:00.628217 18178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:00.659312 18178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "3fee3852f9e046febe96e7d8d6783ea4"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "5ffa83f013bc5c49ed543d2d3cd78ba3"
server_key_iv: "be214743b5947dcb07b49fce88c21b19"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.659827 18178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "3fee3852f9e046febe96e7d8d6783ea4"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "5ffa83f013bc5c49ed543d2d3cd78ba3"
server_key_iv: "be214743b5947dcb07b49fce88c21b19"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.664897 18178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260808 00:35:00.686539 18198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.686811 18178 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:35:00.686905 18178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "3fee3852f9e046febe96e7d8d6783ea4"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "5ffa83f013bc5c49ed543d2d3cd78ba3"
server_key_iv: "be214743b5947dcb07b49fce88c21b19"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.686998 18178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:00.700996 18178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:00.703404 18178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:00.703591 18178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:00.703867 18178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:00.704293 18178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:00.704349 18178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.704419 18178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:00.704449 18178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.711548 18178 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:45083
I20260808 00:35:00.712119 18178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:35:00.713498 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 18178
I20260808 00:35:00.713626 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:35:00.713896 32587 external_mini_cluster.cc:1428] Setting key 75d0a9da39967663c77e170716fda189
I20260808 00:35:00.715732 18362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:45083 every 8 connection(s)
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:00.727571 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:00.716575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:55947 (local address 127.31.210.254:46157)
0808 00:35:00.716626 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:00.716628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:00.717006 (+ 378us) server_negotiation.cc:407] Connection header received
0808 00:35:00.717718 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:00.717720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:00.717734 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:00.717764 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:00.718676 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.719476 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.721096 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.721461 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.721570 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:00.724244 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:00.724246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:00.724246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:00.724255 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:00.726112 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:00.726529 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:00.726531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:00.726531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:00.726606 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:00.726864 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:00.726865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:00.726866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:00.727024 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:00.727102 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:00.727369 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 00:35:00.727475 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:35:00.727725 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:00.716571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46157 (local address 127.31.210.195:55947)
0808 00:35:00.716841 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:00.716884 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:00.717532 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:00.717968 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:00.717972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:00.718236 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:00.718535 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:00.718542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.719613 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.719614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:00.720987 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:00.720991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:00.721609 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:00.721610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:00.721766 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:00.722516 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:00.722528 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:00.724123 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:00.726201 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:00.726208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:00.726210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:00.726449 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:00.726678 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:00.726679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:00.726679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:00.726794 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:00.727094 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:00.727099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:00.727229 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 00:35:00.727317 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:35:00.727445 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260808 00:35:00.728324 18363 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46157
I20260808 00:35:00.728461 18363 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:00.728742 18363 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:00.729208 17647 ts_manager.cc:194] Registered new tserver with Master: 3fee3852f9e046febe96e7d8d6783ea4 (127.31.210.195:45083)
I20260808 00:35:00.729657 17647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:55947
I20260808 00:35:00.736958 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:35:00.737740 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 17769
I20260808 00:35:00.758749 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 17977
I20260808 00:35:00.783537 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 18178
I20260808 00:35:00.790964 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 17601
I20260808 00:35:00.809325 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46157
--webserver_interface=127.31.210.254
--webserver_port=32881
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46157
--ranger_config_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:00.898566 18366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:00.898785 18366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:00.898808 18366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:00.899914 18366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:00.899951 18366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:00.899971 18366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:00.899984 18366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:00.899996 18366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:00.902164 18366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46157
--ranger_config_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=32881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:00.902415 18366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:00.902628 18366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:00.908313 18379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.908371 18376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:00.908317 18375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:00.908591 18366 server_base.cc:1034] running on GCE node
I20260808 00:35:00.908912 18366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:00.909265 18366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:00.910460 18366 hybrid_clock.cc:648] HybridClock initialized: now 1786149300910439 us; error 34 us; skew 500 ppm
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:00.913627 18366 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:00.914187 18366 webserver.cc:460] Webserver started at http://127.31.210.254:32881/ using document root <none> and password file <none>
I20260808 00:35:00.914461 18366 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:00.914549 18366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:35:00 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149300, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:00.964802 18366 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:35:00.965698 18390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:00.965937 18366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:00.966133 18366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "ffc4317c271c45ec9b04a29a6d16d677"
format_stamp: "Formatted at 2026-08-08 00:34:58 on dist-test-slave-44d2"
server_key: "6756d4c02086a2f6417f590907b735c9"
server_key_iv: "c1a8d42bff32944e09a9fdedfedfdabe"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:00.966570 18366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:00.978168 18366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:00.980405 18366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:00.980628 18366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:00.985554 18366 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46157
I20260808 00:35:00.985621 18468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46157 every 8 connection(s)
I20260808 00:35:00.986039 18366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:00.987286 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 18366
I20260808 00:35:00.987493 18469 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:35:00.987646 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:34731
--local_ip_for_outbound_sockets=127.31.210.193
--tserver_master_addrs=127.31.210.254:46157
--webserver_port=33885
--webserver_interface=127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:00.988356 18469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Bootstrap starting.
I20260808 00:35:01.000161 18469 log.cc:824] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:01.003183 18469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:35:01.003629 18469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Bootstrap complete.
I20260808 00:35:01.007705 18469 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:35:01.008106 18469 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc4317c271c45ec9b04a29a6d16d677, State: Initialized, Role: FOLLOWER
I20260808 00:35:01.008301 18469 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:35:01.008430 18469 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:35:01.008522 18469 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:35:01.008574 18469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:35:01.010149 18469 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:35:01.010290 18469 leader_election.cc:302] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc4317c271c45ec9b04a29a6d16d677; no voters:
I20260808 00:35:01.010501 18469 leader_election.cc:288] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:35:01.010613 18473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:35:01.010825 18473 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [term 2 LEADER]: Becoming Leader. State: Replica: ffc4317c271c45ec9b04a29a6d16d677, State: Running, Role: LEADER
I20260808 00:35:01.010890 18469 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:01.011020 18473 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } }
I20260808 00:35:01.011478 18474 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ffc4317c271c45ec9b04a29a6d16d677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } } }
I20260808 00:35:01.011588 18474 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:01.011489 18475 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc4317c271c45ec9b04a29a6d16d677. Latest consensus state: current_term: 2 leader_uuid: "ffc4317c271c45ec9b04a29a6d16d677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc4317c271c45ec9b04a29a6d16d677" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46157 } } }
I20260808 00:35:01.011665 18475 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677 [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:01.011934 18486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:01.012660 18486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:01.012851 18486 catalog_manager.cc:1259] Loaded cluster ID: 76304353827848a09b8b9dbca1fd1648
I20260808 00:35:01.012956 18486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:35:01.013258 18469 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:35:01.017927 18486 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:35:01.018592 18486 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Loaded TSK: 0
I20260808 00:35:01.019034 18486 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:35:01.127739 18471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:01.128141 18471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:01.128328 18471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:01.130697 18471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:01.130784 18471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:01.130834 18471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:35:01.133642 18471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:34731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=33885
--tserver_master_addrs=127.31.210.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:01.134126 18471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:01.134456 18471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:01.142066 18531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:01.142515 18471 server_base.cc:1034] running on GCE node
W20260808 00:35:01.142659 18529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:01.144145 18528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:01.144546 18471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:01.144892 18471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:01.146097 18471 hybrid_clock.cc:648] HybridClock initialized: now 1786149301146069 us; error 42 us; skew 500 ppm
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:01.150053 18471 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:35:01.150735 18471 webserver.cc:460] Webserver started at http://127.31.210.193:33885/ using document root <none> and password file <none>
I20260808 00:35:01.151059 18471 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:01.151165 18471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:01.176172 18471 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 00:35:01.177644 18545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.177911 18471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:35:01.178009 18471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "f1f024e9b9c44976a085537f8c7ec8ae"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "9d1c4bb8ee9753deda61e0259d65493a"
server_key_iv: "e667a5ee2021da31f97a92fd947b4842"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:01.178536 18471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:01.191208 18471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:01.194252 18471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:01.194442 18471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:01.194706 18471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:01.195133 18471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:01.195191 18471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.195227 18471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:01.195291 18471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.204003 18471 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:34731
I20260808 00:35:01.204102 18714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:34731 every 8 connection(s)
I20260808 00:35:01.204847 18471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:35:01.206985 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 18471
I20260808 00:35:01.207316 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:34295
--local_ip_for_outbound_sockets=127.31.210.194
--tserver_master_addrs=127.31.210.254:46157
--webserver_port=35559
--webserver_interface=127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:01.223469 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:01.205736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46157 (local address 127.31.210.193:50041)
0808 00:35:01.206747 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:01.208081 (+ 1334us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:01.208817 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:01.209632 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:01.209636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:01.209920 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:01.210234 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:01.210241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.212122 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.212123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:01.213591 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:01.213595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.214702 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.214703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:01.214857 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:01.216143 (+ 1286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:01.216156 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:01.218384 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:01.221140 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:01.221149 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:01.221152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:01.221533 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:01.221882 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:01.221883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:01.221884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:01.222026 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:01.222571 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:01.222578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:01.222747 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 00:35:01.222851 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:35:01.223193 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":103,"threads_started":1}
I20260808 00:35:01.224117 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:01.205831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:50041 (local address 127.31.210.254:46157)
0808 00:35:01.206110 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:01.206121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:01.209015 (+ 2894us) server_negotiation.cc:407] Connection header received
0808 00:35:01.209181 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:01.209189 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:01.209250 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:01.209444 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:01.210391 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.211976 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.213711 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.214420 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.214555 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:01.218568 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:01.218574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:01.218575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:01.218590 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:01.221015 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.221641 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.221645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.221645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.221796 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.222108 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.222109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.222110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.222450 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:01.222538 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:01.223719 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0808 00:35:01.223864 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":141,"threads_started":1}
I20260808 00:35:01.225004 18717 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46157
I20260808 00:35:01.225116 18717 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.225343 18717 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
W20260808 00:35:01.227047 18717 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:46157 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:35:01.227134 18717 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.227209 18717 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:01.227734 18717 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.227815 18717 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
W20260808 00:35:01.228250 18717 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:35:01.342796 18721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:01.343201 18721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:01.343250 18721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:01.345309 18721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:01.345445 18721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:01.345562 18721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:35:01.347798 18721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:34295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=35559
--tserver_master_addrs=127.31.210.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:01.348093 18721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:01.348362 18721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:01.355226 18728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:01.355242 18727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:01.355453 18721 server_base.cc:1034] running on GCE node
W20260808 00:35:01.355508 18730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:01.355697 18721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:01.356105 18721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:01.357270 18721 hybrid_clock.cc:648] HybridClock initialized: now 1786149301357136 us; error 149 us; skew 500 ppm
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:01.360731 18721 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:35:01.361305 18721 webserver.cc:460] Webserver started at http://127.31.210.194:35559/ using document root <none> and password file <none>
I20260808 00:35:01.361559 18721 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:01.361649 18721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:01.385179 18721 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 00:35:01.386348 18744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.386583 18721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:35:01.386674 18721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "f146e2ee35f046c9afbd281b7bcff40c"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "ec4aff9c8110e2242716776c2fb6209d"
server_key_iv: "c2ed107a0824f64e5338063c64fdb2a2"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:01.387151 18721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:01.423144 18721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:01.425503 18721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:01.425685 18721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:01.425954 18721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:01.426348 18721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:01.426402 18721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.427052 18721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:01.427119 18721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.435211 18721 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:34295
I20260808 00:35:01.435319 18926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:34295 every 8 connection(s)
I20260808 00:35:01.435742 18721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:35:01.438328 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 18721
I20260808 00:35:01.438617 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.210.212:40807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:45083
--local_ip_for_outbound_sockets=127.31.210.195
--tserver_master_addrs=127.31.210.254:46157
--webserver_port=35707
--webserver_interface=127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:01.447238 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:01.436820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:53301 (local address 127.31.210.254:46157)
0808 00:35:01.436890 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:01.436892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:01.437147 (+ 255us) server_negotiation.cc:407] Connection header received
0808 00:35:01.437718 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:01.437720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:01.437733 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:01.437803 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:01.438454 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.439074 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.440536 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.440760 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.440845 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:01.443354 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:01.443356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:01.443357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:01.443365 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:01.445232 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.445838 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.445839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.445839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.445919 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.446278 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.446280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.446280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.446427 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:01.446513 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:01.446949 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 00:35:01.447064 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:35:01.447904 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:01.436780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46157 (local address 127.31.210.194:53301)
0808 00:35:01.437000 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:01.437044 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:01.437599 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:01.437894 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:01.437897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:01.438114 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:01.438327 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:01.438331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.439190 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.439192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:01.440406 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:01.440409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.440951 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.440952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:01.441067 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:01.441649 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:01.441659 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:01.443240 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:01.445330 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:01.445338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:01.445341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:01.445736 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:01.446016 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:01.446017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:01.446018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:01.446191 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:01.446514 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:01.446522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:01.446721 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 00:35:01.447368 (+ 647us) client_negotiation.cc:240] Negotiation successful
0808 00:35:01.447576 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":111,"threads_started":1}
I20260808 00:35:01.448885 18929 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46157
I20260808 00:35:01.449004 18929 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.449244 18929 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
W20260808 00:35:01.450052 18929 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:46157 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:35:01.450132 18929 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.450206 18929 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:01.450631 18929 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.450719 18929 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
W20260808 00:35:01.451130 18929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:35:01.564371 18935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:01.564697 18935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:01.564746 18935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:01.566047 18935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:01.566120 18935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:01.566167 18935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:35:01.567739 18935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.210.212:40807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:45083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=35707
--tserver_master_addrs=127.31.210.254:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:01.568027 18935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:01.568277 18935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:01.574522 18944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:01.574564 18947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:01.574765 18943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:01.575054 18935 server_base.cc:1034] running on GCE node
I20260808 00:35:01.575314 18935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:01.575637 18935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:01.576817 18935 hybrid_clock.cc:648] HybridClock initialized: now 1786149301576798 us; error 26 us; skew 500 ppm
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:01.580179 18935 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:35:01.580945 18935 webserver.cc:460] Webserver started at http://127.31.210.195:35707/ using document root <none> and password file <none>
I20260808 00:35:01.581290 18935 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:01.581408 18935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:01.595645 18935 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:01.596889 18963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.597940 18935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 00:35:01.598074 18935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "3fee3852f9e046febe96e7d8d6783ea4"
format_stamp: "Formatted at 2026-08-08 00:35:00 on dist-test-slave-44d2"
server_key: "5ffa83f013bc5c49ed543d2d3cd78ba3"
server_key_iv: "be214743b5947dcb07b49fce88c21b19"
server_key_version: "kuduclusterkey@0"
I20260808 00:35:01.598587 18935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:01.613845 18935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:01.616706 18935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:01.616981 18935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:01.617358 18935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:01.617831 18935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:01.617931 18935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.617969 18935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:01.618005 18935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:01.626789 18935 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:45083
I20260808 00:35:01.626876 19149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:45083 every 8 connection(s)
I20260808 00:35:01.627336 18935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:35:01.628743 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 18935
Aug 08 00:35:01 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149301, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:01.641686 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:01.628244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46157 (local address 127.31.210.195:45179)
0808 00:35:01.628434 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:01.628591 (+ 157us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:01.629469 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:01.629803 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:01.629808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:01.630042 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:01.630247 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:01.630252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.631112 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.631113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:01.633411 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:01.633414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.634032 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.634034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:01.634191 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:01.634946 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:01.634957 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:01.637088 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:01.639890 (+ 2802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:01.639899 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:01.639901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:01.640190 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:01.640404 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:01.640405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:01.640405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:01.640585 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:01.640855 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:01.640861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:01.641047 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 00:35:01.641258 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 00:35:01.641430 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":108,"threads_started":1}
I20260808 00:35:01.644786 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:01.628226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:45179 (local address 127.31.210.254:46157)
0808 00:35:01.628278 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:01.628280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:01.628689 (+ 409us) server_negotiation.cc:407] Connection header received
0808 00:35:01.629595 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:01.629596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:01.629609 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:01.629658 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:01.630347 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.631001 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.633544 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.633853 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.633905 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:01.637264 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:01.637266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:01.637267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:01.637276 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:01.639829 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.640289 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.640290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.640290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.640339 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.640666 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.640667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.640667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.640775 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:01.640818 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:01.641217 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 00:35:01.644680 (+ 3463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3370,"server-negotiator.queue_time_us":21}
I20260808 00:35:01.645555 19150 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46157
I20260808 00:35:01.645661 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:01.629291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.31.210.254:46157)
0808 00:35:01.629670 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:01.629671 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:01.629679 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:35:01.629696 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:01.629697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:01.629708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:01.629780 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:01.630697 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.631302 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.634911 (+ 3609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:01.635209 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:01.635282 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:01.637786 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:01.637787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:01.637787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:01.637796 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:01.639719 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.640185 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.640186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.640187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.640256 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:01.640572 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:01.640573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:01.640573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:01.640723 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:01.640899 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:01.641004 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:35:01.645524 (+ 4520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":63,"threads_started":1}
I20260808 00:35:01.645676 19150 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.645901 19150 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
W20260808 00:35:01.646908 19150 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:46157 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:35:01.647007 19150 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.647095 19150 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:01.647567 19150 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:01.647658 19150 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
W20260808 00:35:01.648012 19150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 00:35:02 dist-test-slave-44d2 krb5kdc[16934](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149302, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:02.228451 18717 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:02.228653 18717 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:02.229036 18416 ts_manager.cc:194] Registered new tserver with Master: f1f024e9b9c44976a085537f8c7ec8ae (127.31.210.193:34731)
I20260808 00:35:02.229763 18416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:50041
I20260808 00:35:02.451321 18929 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:02.451494 18929 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:02.451792 18416 ts_manager.cc:194] Registered new tserver with Master: f146e2ee35f046c9afbd281b7bcff40c (127.31.210.194:34295)
I20260808 00:35:02.452339 18416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:53301
Aug 08 00:35:02 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149302, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:02.648237 19150 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:02.648615 19150 heartbeater.cc:511] Master 127.31.210.254:46157 requested a full tablet report, sending...
I20260808 00:35:02.651118 18416 ts_manager.cc:194] Registered new tserver with Master: 3fee3852f9e046febe96e7d8d6783ea4 (127.31.210.195:45083)
I20260808 00:35:02.651774 18416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:45179
I20260808 00:35:02.664060 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:35:02.689952 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.672799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.31.210.254:46157)
0808 00:35:02.675068 (+ 2269us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:02.675071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:02.675279 (+ 208us) server_negotiation.cc:407] Connection header received
0808 00:35:02.675306 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:02.675307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:02.675319 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:02.675395 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:02.676220 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.677073 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.678133 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.678467 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.678540 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.686114 (+ 7574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:02.686116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:02.686117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:02.686129 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:02.688378 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:02.688882 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:02.688884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:02.688884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:02.688973 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:02.689223 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:02.689224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:02.689224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:02.689402 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:02.689528 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:02.689698 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 00:35:02.689808 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2241,"thread_start_us":82,"threads_started":1}
I20260808 00:35:02.692691 18416 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:35:02.695261 18416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:35:02.708865 19202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.703110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:34295 (local address 127.0.0.1:40050)
0808 00:35:02.704076 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:02.704109 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:02.704173 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:02.704557 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:02.704560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:02.704563 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:02.704641 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.704643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.706142 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.706143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.707386 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.707390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.708562 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.708563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.708646 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.708662 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:35:02.708712 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:35:02.708765 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":103,"threads_started":1}
I20260808 00:35:02.709514 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.703231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.31.210.194:34295)
0808 00:35:02.704337 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:02.704341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:02.704351 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:35:02.704387 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:02.704389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:02.704405 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:02.704734 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:02.704815 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.706033 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.707508 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.708417 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.709269 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.709358 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:02.709388 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:35:02.709431 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":66,"threads_started":1}
I20260808 00:35:02.711359 18797 tablet_service.cc:1467] Processing CreateTablet for tablet 8d86da3ce3c84a3da7f6efab9e9728c6 (DEFAULT_TABLE table=test-table [id=d89c775097b44c4983e7e0994f948ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:02.711664 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.703367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.31.210.193:34731)
0808 00:35:02.704040 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:02.704045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:02.704316 (+ 271us) server_negotiation.cc:407] Connection header received
0808 00:35:02.704369 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:02.704372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:02.704392 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:02.704508 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:02.704822 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.706620 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.709847 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.710720 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.710804 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.710863 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:02.711537 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 00:35:02.711586 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":67,"threads_started":1}
I20260808 00:35:02.711458 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.703052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:34731 (local address 127.0.0.1:36988)
0808 00:35:02.704046 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:02.704093 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:02.704174 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:02.704646 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:02.704647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:02.704651 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:02.704704 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.704706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.706745 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.706746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.708043 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.708046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.711220 (+ 3174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.711221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.711278 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.711283 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:02.711343 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:35:02.711385 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":87,"threads_started":1}
I20260808 00:35:02.711843 18797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d86da3ce3c84a3da7f6efab9e9728c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:02.714393 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.703526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.31.210.195:45083)
0808 00:35:02.703785 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:02.703790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:02.708653 (+ 4863us) server_negotiation.cc:407] Connection header received
0808 00:35:02.708797 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:02.708800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:02.708827 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:02.708904 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:02.709351 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.710546 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.713085 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.713874 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.713939 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.714010 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:02.714262 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 00:35:02.714308 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":137,"threads_started":1}
I20260808 00:35:02.714632 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.703365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45083 (local address 127.0.0.1:49104)
0808 00:35:02.708563 (+ 5198us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:02.708577 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:02.708689 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:02.709166 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:02.709167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:02.709169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:02.709231 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.709233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.711018 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.711019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.712970 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.712973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.714080 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.714081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.714155 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.714161 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:35:02.714511 (+ 350us) client_negotiation.cc:240] Negotiation successful
0808 00:35:02.714558 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5171,"thread_start_us":155,"threads_started":1}
I20260808 00:35:02.715448 19042 tablet_service.cc:1467] Processing CreateTablet for tablet 8d86da3ce3c84a3da7f6efab9e9728c6 (DEFAULT_TABLE table=test-table [id=d89c775097b44c4983e7e0994f948ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:02.715857 19042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d86da3ce3c84a3da7f6efab9e9728c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:02.715752 18589 tablet_service.cc:1467] Processing CreateTablet for tablet 8d86da3ce3c84a3da7f6efab9e9728c6 (DEFAULT_TABLE table=test-table [id=d89c775097b44c4983e7e0994f948ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:02.716176 18589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d86da3ce3c84a3da7f6efab9e9728c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:02.717301 19206 tablet_bootstrap.cc:492] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c: Bootstrap starting.
I20260808 00:35:02.718269 19206 tablet_bootstrap.cc:654] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:02.719246 19206 log.cc:824] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:02.719650 19208 tablet_bootstrap.cc:492] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Bootstrap starting.
I20260808 00:35:02.720466 19208 tablet_bootstrap.cc:654] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:02.721014 19212 tablet_bootstrap.cc:492] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae: Bootstrap starting.
I20260808 00:35:02.721071 19208 log.cc:824] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:02.721889 19212 tablet_bootstrap.cc:654] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:02.722131 19206 tablet_bootstrap.cc:492] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c: No bootstrap required, opened a new log
I20260808 00:35:02.722319 19206 ts_tablet_manager.cc:1397] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 00:35:02.722441 19212 log.cc:824] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:02.723434 19212 tablet_bootstrap.cc:492] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae: No bootstrap required, opened a new log
I20260808 00:35:02.723531 19212 ts_tablet_manager.cc:1397] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:35:02.726281 19206 raft_consensus.cc:348] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.726483 19206 raft_consensus.cc:374] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:02.726531 19206 raft_consensus.cc:729] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f146e2ee35f046c9afbd281b7bcff40c, State: Initialized, Role: FOLLOWER
I20260808 00:35:02.726635 19212 raft_consensus.cc:348] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.726702 19206 consensus_queue.cc:260] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.726794 19212 raft_consensus.cc:374] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:02.726835 19212 raft_consensus.cc:729] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1f024e9b9c44976a085537f8c7ec8ae, State: Initialized, Role: FOLLOWER
I20260808 00:35:02.727056 19206 ts_tablet_manager.cc:1428] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 00:35:02.727211 18929 heartbeater.cc:503] Master 127.31.210.254:46157 was elected leader, sending a full tablet report...
I20260808 00:35:02.727463 19212 consensus_queue.cc:260] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.727805 19212 ts_tablet_manager.cc:1428] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 00:35:02.727931 18717 heartbeater.cc:503] Master 127.31.210.254:46157 was elected leader, sending a full tablet report...
I20260808 00:35:02.730017 19208 tablet_bootstrap.cc:492] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: No bootstrap required, opened a new log
I20260808 00:35:02.730135 19208 ts_tablet_manager.cc:1397] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260808 00:35:02.732308 19208 raft_consensus.cc:348] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.732502 19208 raft_consensus.cc:374] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:02.732643 19208 raft_consensus.cc:729] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fee3852f9e046febe96e7d8d6783ea4, State: Initialized, Role: FOLLOWER
I20260808 00:35:02.732817 19208 consensus_queue.cc:260] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.733150 19208 ts_tablet_manager.cc:1428] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 00:35:02.733316 19150 heartbeater.cc:503] Master 127.31.210.254:46157 was elected leader, sending a full tablet report...
I20260808 00:35:02.806531 19216 raft_consensus.cc:482] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:02.806809 19216 raft_consensus.cc:504] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.807442 19216 leader_election.cc:288] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f146e2ee35f046c9afbd281b7bcff40c (127.31.210.194:34295), 3fee3852f9e046febe96e7d8d6783ea4 (127.31.210.195:45083)
I20260808 00:35:02.812093 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.808336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:33827 (local address 127.31.210.195:45083)
0808 00:35:02.808388 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:02.808390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:02.808398 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:35:02.808426 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:02.808427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:02.808437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:02.808515 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:02.808749 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.809506 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.811068 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.811631 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.811686 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.811696 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:02.811961 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 00:35:02.812011 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:35:02.812644 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.807958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:45083 (local address 127.31.210.193:33827)
0808 00:35:02.808091 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:02.808100 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:02.808146 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:02.808563 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:02.808563 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:02.808565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:02.808650 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.808652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.809628 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.809628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.810930 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.810935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.811788 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.811789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.811863 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.811868 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:02.812431 (+ 563us) client_negotiation.cc:240] Negotiation successful
0808 00:35:02.812512 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260808 00:35:02.812867 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.808174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:54083 (local address 127.31.210.194:34295)
0808 00:35:02.808314 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:02.808316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:02.808327 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:35:02.808372 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:02.808373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:02.808385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:02.808451 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:02.808749 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.809514 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.811818 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.812399 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.812456 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.812464 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:02.812748 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 00:35:02.812801 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260808 00:35:02.813261 19218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.807652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:34295 (local address 127.31.210.193:54083)
0808 00:35:02.807878 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:02.807892 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:02.807954 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:02.808550 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:02.808551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:02.808554 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:02.808653 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.808655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.809628 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.809629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.811700 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:02.811703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.812555 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.812556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:02.812643 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.812647 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:35:02.813128 (+ 481us) client_negotiation.cc:240] Negotiation successful
0808 00:35:02.813172 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":1017,"thread_start_us":146,"threads_started":1}
I20260808 00:35:02.813359 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d86da3ce3c84a3da7f6efab9e9728c6" candidate_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fee3852f9e046febe96e7d8d6783ea4" is_pre_election: true
I20260808 00:35:02.813555 19078 raft_consensus.cc:2454] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1f024e9b9c44976a085537f8c7ec8ae in term 0.
I20260808 00:35:02.813845 18546 leader_election.cc:302] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fee3852f9e046febe96e7d8d6783ea4, f1f024e9b9c44976a085537f8c7ec8ae; no voters:
I20260808 00:35:02.814296 19216 raft_consensus.cc:2781] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:35:02.814373 19216 raft_consensus.cc:482] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:35:02.814402 19216 raft_consensus.cc:3037] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:02.814342 18861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d86da3ce3c84a3da7f6efab9e9728c6" candidate_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f146e2ee35f046c9afbd281b7bcff40c" is_pre_election: true
I20260808 00:35:02.814517 18861 raft_consensus.cc:2454] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1f024e9b9c44976a085537f8c7ec8ae in term 0.
I20260808 00:35:02.815435 19216 raft_consensus.cc:504] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.815593 19216 leader_election.cc:288] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [CANDIDATE]: Term 1 election: Requested vote from peers f146e2ee35f046c9afbd281b7bcff40c (127.31.210.194:34295), 3fee3852f9e046febe96e7d8d6783ea4 (127.31.210.195:45083)
I20260808 00:35:02.816195 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d86da3ce3c84a3da7f6efab9e9728c6" candidate_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fee3852f9e046febe96e7d8d6783ea4"
I20260808 00:35:02.816313 19078 raft_consensus.cc:3037] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:02.817482 19078 raft_consensus.cc:2454] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1f024e9b9c44976a085537f8c7ec8ae in term 1.
I20260808 00:35:02.817691 18546 leader_election.cc:302] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fee3852f9e046febe96e7d8d6783ea4, f1f024e9b9c44976a085537f8c7ec8ae; no voters:
I20260808 00:35:02.817670 18861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d86da3ce3c84a3da7f6efab9e9728c6" candidate_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f146e2ee35f046c9afbd281b7bcff40c"
I20260808 00:35:02.817765 18861 raft_consensus.cc:3037] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:02.817870 19216 raft_consensus.cc:2781] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:02.818040 19216 raft_consensus.cc:686] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [term 1 LEADER]: Becoming Leader. State: Replica: f1f024e9b9c44976a085537f8c7ec8ae, State: Running, Role: LEADER
I20260808 00:35:02.818133 19216 consensus_queue.cc:237] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } }
I20260808 00:35:02.818819 18861 raft_consensus.cc:2454] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1f024e9b9c44976a085537f8c7ec8ae in term 1.
I20260808 00:35:02.819307 18416 catalog_manager.cc:5495] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae reported cstate change: term changed from 0 to 1, leader changed from <none> to f1f024e9b9c44976a085537f8c7ec8ae (127.31.210.193). New cstate: current_term: 1 leader_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1f024e9b9c44976a085537f8c7ec8ae" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 } health_report { overall_health: UNKNOWN } } }
I20260808 00:35:02.828517 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.825343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.31.210.193:34731)
0808 00:35:02.825442 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:02.825445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:02.825453 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:35:02.825491 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:02.825492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:02.825513 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:02.825583 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:35:02.825798 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.826343 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.827178 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:02.827467 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:02.827550 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:02.827780 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:35:02.827950 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:35:02.828049 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:02.828384 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 00:35:02.828408 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 00:35:02.830852 18861 raft_consensus.cc:1261] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 1 FOLLOWER]: Refusing update from remote peer f1f024e9b9c44976a085537f8c7ec8ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:02.830946 19078 raft_consensus.cc:1261] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 1 FOLLOWER]: Refusing update from remote peer f1f024e9b9c44976a085537f8c7ec8ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:02.831229 19220 consensus_queue.cc:1035] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f146e2ee35f046c9afbd281b7bcff40c" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 34295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:02.831722 19220 consensus_queue.cc:1035] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f1f024e9b9c44976a085537f8c7ec8ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fee3852f9e046febe96e7d8d6783ea4" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 45083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:02.837211 19225 mvcc.cc:205] Tried to move back new op lower bound from 7316067544392368128 to 7316067544343822336. Current Snapshot: MvccSnapshot[applied={T|T < 7316067544392368128}]
I20260808 00:35:02.840176 19227 mvcc.cc:205] Tried to move back new op lower bound from 7316067544392368128 to 7316067544343822336. Current Snapshot: MvccSnapshot[applied={T|T < 7316067544392368128 or (T in {7316067544392368128})}]
I20260808 00:35:02.841965 19232 mvcc.cc:205] Tried to move back new op lower bound from 7316067544392368128 to 7316067544343822336. Current Snapshot: MvccSnapshot[applied={T|T < 7316067544392368128}]
I20260808 00:35:02.842388 18416 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:02.842506 18416 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:02.843331 18416 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffc4317c271c45ec9b04a29a6d16d677: Sending DeleteTablet for 3 replicas of tablet 8d86da3ce3c84a3da7f6efab9e9728c6
I20260808 00:35:02.843798 18797 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d86da3ce3c84a3da7f6efab9e9728c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:02 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:40050
I20260808 00:35:02.843930 18589 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d86da3ce3c84a3da7f6efab9e9728c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:02 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:36988
I20260808 00:35:02.844304 19042 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d86da3ce3c84a3da7f6efab9e9728c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:02 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:49104
I20260808 00:35:02.844606 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 18471
I20260808 00:35:02.854521 19240 tablet_replica.cc:330] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c: stopping tablet replica
I20260808 00:35:02.854719 19240 raft_consensus.cc:2229] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:02.854894 19240 raft_consensus.cc:2258] T 8d86da3ce3c84a3da7f6efab9e9728c6 P f146e2ee35f046c9afbd281b7bcff40c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:02.855284 19244 tablet_replica.cc:330] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: stopping tablet replica
I20260808 00:35:02.855456 19244 raft_consensus.cc:2229] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:02.855594 19244 raft_consensus.cc:2258] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:02.856186 19244 ts_tablet_manager.cc:1905] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:02.858553 19244 ts_tablet_manager.cc:1918] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:02.858729 19244 log.cc:1196] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/8d86da3ce3c84a3da7f6efab9e9728c6
I20260808 00:35:02.859181 19244 ts_tablet_manager.cc:1939] T 8d86da3ce3c84a3da7f6efab9e9728c6 P 3fee3852f9e046febe96e7d8d6783ea4: Deleting consensus metadata
W20260808 00:35:02.859865 18396 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:34731 (error 108)
I20260808 00:35:02.860183 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 18721
I20260808 00:35:02.865193 18392 catalog_manager.cc:4841] TS 3fee3852f9e046febe96e7d8d6783ea4 (127.31.210.195:45083): tablet 8d86da3ce3c84a3da7f6efab9e9728c6 (table test-table [id=d89c775097b44c4983e7e0994f948ab9]) successfully deleted
I20260808 00:35:02.867051 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.866884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:34731 (local address 127.0.0.1:37004)
0808 00:35:02.866954 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:02.867008 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:34731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 00:35:02.867343 18396 catalog_manager.cc:4584] TS f1f024e9b9c44976a085537f8c7ec8ae (127.31.210.193:34731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d86da3ce3c84a3da7f6efab9e9728c6: Network error: Client connection negotiation failed: client connection to 127.31.210.193:34731: connect: Connection refused (error 111)
Aug 08 00:35:02 dist-test-slave-44d2 krb5kdc[16934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149302, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for HTTP/127.31.210.212@KRBTEST.COM
I20260808 00:35:02.872938 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:02.872806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:34295 (local address 127.0.0.1:40062)
0808 00:35:02.872868 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:02.872898 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.194:34295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:35:02.873071 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 18935
I20260808 00:35:02.883486 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 18366
I20260808 00:35:02.900960 32587 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 00:35:03.256044 32587 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 00:35:03.256145 32587 mini_ranger.cc:66] Stopping Ranger...
I20260808 00:35:03.611099 32587 mini_ranger.cc:68] Stopped Ranger
2026-08-08 00:35:03.611 UTC [16981] LOG: received smart shutdown request
2026-08-08 00:35:03.616 UTC [16981] LOG: background worker "logical replication launcher" (PID 16994) exited with exit code 1
2026-08-08 00:35:03.616 UTC [16986] LOG: shutting down
2026-08-08 00:35:03.800 UTC [16981] LOG: database system is shut down
2026-08-08T00:35:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52243 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:35:03 dist-test-slave-44d2 krb5kdc[19258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:35:03 dist-test-slave-44d2 krb5kdc[19258](info): set up 2 sockets
Aug 08 00:35:03 dist-test-slave-44d2 krb5kdc[19258](info): commencing operation
krb5kdc: starting...
W20260808 00:35:05.193063 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:35:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:35:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:35:05.323546 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:36565
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:36565,127.31.210.253:37061,127.31.210.252:41057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:05.411144 19276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:05.411370 19276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:05.411398 19276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:05.412518 19276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:05.412556 19276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:05.412572 19276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:05.412593 19276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:05.412626 19276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:05.414209 19276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:36565,127.31.210.253:37061,127.31.210.252:41057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:36565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:05.414426 19276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:05.414606 19276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:05.420765 19284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:05.420771 19296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:05.420795 19283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:05.421276 19276 server_base.cc:1034] running on GCE node
I20260808 00:35:05.421545 19276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:05.421790 19276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:05.422942 19276 hybrid_clock.cc:648] HybridClock initialized: now 1786149305422926 us; error 32 us; skew 500 ppm
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:05.425645 19276 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:05.426162 19276 webserver.cc:460] Webserver started at http://127.31.210.254:40471/ using document root <none> and password file <none>
I20260808 00:35:05.426390 19276 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:05.426457 19276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:05.426604 19276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:05.427657 19276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "a7ee15ae82eb4dbfa40329d6d476cb40"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "64e9e448e01aae234a8e353a8cc0a034"
server_key_iv: "b9ec0ed5bedd8db0270e011f6b9f6d82"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.428032 19276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "a7ee15ae82eb4dbfa40329d6d476cb40"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "64e9e448e01aae234a8e353a8cc0a034"
server_key_iv: "b9ec0ed5bedd8db0270e011f6b9f6d82"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.429503 19276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:35:05.430370 19304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:05.430539 19276 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:35:05.430644 19276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "a7ee15ae82eb4dbfa40329d6d476cb40"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "64e9e448e01aae234a8e353a8cc0a034"
server_key_iv: "b9ec0ed5bedd8db0270e011f6b9f6d82"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.430717 19276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:05.440620 19276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:05.442840 19276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:05.443008 19276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:05.447289 19276 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:36565
I20260808 00:35:05.447324 19381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:36565 every 8 connection(s)
I20260808 00:35:05.447681 19276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:05.448572 19382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:05.448928 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 19276
I20260808 00:35:05.449105 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:35:05.449291 32587 external_mini_cluster.cc:1428] Setting key 4ec3ce62ca30840960a41f10a6ea8a1e
I20260808 00:35:05.450225 19382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.210.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.253@KRBTEST.COM" created.
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:05.466245 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.451024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.31.210.254:36565)
0808 00:35:05.451783 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.451786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.451796 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:35:05.451854 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.451855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.451869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.451963 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.456648 (+ 4685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.457578 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.458659 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.459020 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.459121 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.462345 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.462352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.462353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.462360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.464375 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.464983 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.464987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.464987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.465084 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.465401 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.465402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.465402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.465640 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.465709 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.465951 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.466073 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":88,"threads_started":1}
I20260808 00:35:05.466403 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.450955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:36565 (local address 127.0.0.1:44824)
0808 00:35:05.451201 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.451234 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.451655 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.451982 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.451985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.452268 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.452412 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.452417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.457697 (+ 5280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.457699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.458558 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.458561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.459112 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.459112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.459172 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.459723 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.459732 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.462240 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.464473 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.464521 (+ 48us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.464523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.464754 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.465177 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.465178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.465179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.465303 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.465714 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.465722 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.465803 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.465899 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.466195 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":82,"thread_start_us":78,"threads_started":1}
I20260808 00:35:05.467320 19382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:35:05.467727 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.467583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48690)
0808 00:35:05.467637 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.467693 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:35:05.467880 19305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
I20260808 00:35:05.468428 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.468312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48702)
0808 00:35:05.468366 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.468399 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:35:05.468568 19382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:37061: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab.
Entry for principal kudu/127.31.210.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab.
I20260808 00:35:05.479121 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:37061
--webserver_interface=127.31.210.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab
--principal=kudu/127.31.210.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:36565,127.31.210.253:37061,127.31.210.252:41057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:05.517820 19382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } attempt: 1
I20260808 00:35:05.518383 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.518225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48706)
0808 00:35:05.518295 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.518345 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 00:35:05.518777 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.518686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48714)
0808 00:35:05.518727 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.518756 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 00:35:05.518894 19382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:37061: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
W20260808 00:35:05.564831 19392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:05.565039 19392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:05.565062 19392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:05.566191 19392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:05.566236 19392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:05.566251 19392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:05.566293 19392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:05.566354 19392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:05.567855 19392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:36565,127.31.210.253:37061,127.31.210.252:41057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab
--principal=kudu/127.31.210.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:37061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.210.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:05.568101 19392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:05.568346 19392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:05.573835 19399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:05.573904 19392 server_base.cc:1034] running on GCE node
W20260808 00:35:05.573848 19403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:05.573840 19401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:05.574391 19392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:05.574697 19392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:05.575855 19392 hybrid_clock.cc:648] HybridClock initialized: now 1786149305575833 us; error 36 us; skew 500 ppm
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:05.578495 19392 init.cc:377] Logged in from keytab as kudu/127.31.210.253@KRBTEST.COM (short username kudu)
I20260808 00:35:05.579013 19392 webserver.cc:460] Webserver started at http://127.31.210.253:40885/ using document root <none> and password file <none>
I20260808 00:35:05.579264 19392 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:05.579332 19392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:05.579474 19392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:05.580649 19392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data/instance:
uuid: "8fc672ee843d48d3a2ef2decd3af6238"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "1fc2e90d2f7b894f47ffeb6e266d6bd2"
server_key_iv: "5066276a4415c3187d3f06185ca04ede"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.581028 19392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/wal/instance:
uuid: "8fc672ee843d48d3a2ef2decd3af6238"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "1fc2e90d2f7b894f47ffeb6e266d6bd2"
server_key_iv: "5066276a4415c3187d3f06185ca04ede"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.582393 19392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:35:05.583185 19411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:05.583372 19392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:35:05.583479 19392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/wal
uuid: "8fc672ee843d48d3a2ef2decd3af6238"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "1fc2e90d2f7b894f47ffeb6e266d6bd2"
server_key_iv: "5066276a4415c3187d3f06185ca04ede"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.583551 19392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:05.587101 19382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } attempt: 2
I20260808 00:35:05.587546 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.587422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48718)
0808 00:35:05.587480 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.587518 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 00:35:05.587898 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.587805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48734)
0808 00:35:05.587848 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.587878 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:35:05.587990 19382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:37061: Network error: Client connection negotiation failed: client connection to 127.31.210.253:37061: connect: Connection refused (error 111)
I20260808 00:35:05.598012 19392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:05.600373 19392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:05.600564 19392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:05.604552 19392 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.253:37061
I20260808 00:35:05.604588 19481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.253:37061 every 8 connection(s)
I20260808 00:35:05.604991 19392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
I20260808 00:35:05.605886 19482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:05.607436 19482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:05.615209 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 19392
I20260808 00:35:05.615332 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-1/wal/instance
I20260808 00:35:05.615577 32587 external_mini_cluster.cc:1428] Setting key 35e8c3270551a3656dd5c1440c4741f8
I20260808 00:35:05.616783 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.608141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.31.210.254:36565)
0808 00:35:05.608205 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.608207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.608394 (+ 187us) server_negotiation.cc:407] Connection header received
0808 00:35:05.608905 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.608906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.608919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.608981 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.609618 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.610120 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.611084 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.611308 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.611414 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.613664 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.613666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.613666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.613674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.615210 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.615710 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.615711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.615712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.615804 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.616116 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.616117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.616117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.616236 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.616294 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.616642 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.616718 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:35:05.617100 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.608098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:36565 (local address 127.0.0.1:44838)
0808 00:35:05.608254 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.608302 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.608814 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.609086 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.609090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.609335 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.609533 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.609536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.610202 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.610203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.610988 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.610989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.611421 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.611422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.611523 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.612073 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.612082 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.613572 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.615313 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.615322 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.615324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.615627 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.615887 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.615889 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.615889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.616044 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.616304 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.616311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.616502 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.616623 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.616804 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260808 00:35:05.617971 19482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.210.252@KRBTEST.COM; defaulting to no policy
I20260808 00:35:05.625556 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.618263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48744)
0808 00:35:05.618419 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.618431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.618477 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.618781 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.618781 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.618924 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.618970 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.618971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.619810 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.619811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.620369 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.620371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.620863 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.620864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.620933 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.621388 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.621393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.622577 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.624387 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.624388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.624388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.624607 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.624823 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.624824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.624825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.624880 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.625173 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.625174 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.625245 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.625321 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.625428 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260808 00:35:05.625658 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.618465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.31.210.253:37061)
0808 00:35:05.618649 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.618653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.618668 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:35:05.618685 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.618687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.618697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.618780 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.619041 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.619723 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.620451 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.620778 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.620863 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.622675 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.622684 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.622684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.622696 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.624297 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.624699 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.624702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.624703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.624758 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.624953 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.624955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.624955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.625094 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.625181 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.625355 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.625510 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260808 00:35:05.626310 19482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:35:05.626719 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.626593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59770)
0808 00:35:05.626627 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.626698 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 00:35:05.626889 19416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.252@KRBTEST.COM" created.
I20260808 00:35:05.627373 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.627285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59784)
0808 00:35:05.627319 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.627344 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 00:35:05.627489 19482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41057: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab.
Entry for principal kudu/127.31.210.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab.
I20260808 00:35:05.643744 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:41057
--webserver_interface=127.31.210.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab
--principal=kudu/127.31.210.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:36565,127.31.210.253:37061,127.31.210.252:41057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:05.676752 19482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } attempt: 1
I20260808 00:35:05.677354 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.677172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59788)
0808 00:35:05.677252 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.677311 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 00:35:05.677798 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.677691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59804)
0808 00:35:05.677740 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.677773 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:35:05.677934 19482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41057: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
I20260808 00:35:05.679140 19382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } attempt: 3
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:35:05.686656 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.679450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48746)
0808 00:35:05.679502 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.679513 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.679580 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.679722 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.679724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.679892 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.679961 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.679964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.680711 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.680712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.681259 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.681262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.681642 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.681643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.681726 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.682246 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.682251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.683485 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.685536 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.685537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.685538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.685717 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.685978 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.685983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.685983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.686052 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.686405 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.686410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.686454 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.686509 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.686589 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:35:05.686839 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.679525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.31.210.253:37061)
0808 00:35:05.679587 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.679589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.679616 (+ 27us) server_negotiation.cc:407] Connection header received
0808 00:35:05.679641 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.679642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.679658 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.679716 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.680049 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.680621 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.681354 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.681569 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.681645 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.683573 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.683575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.683575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.683584 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.685435 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.685790 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.685791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.685792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.685863 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.686119 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.686120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.686121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.686280 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.686412 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.686558 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.686659 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:35:05.687297 19382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:35:05.687674 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.687575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59806)
0808 00:35:05.687621 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.687652 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 00:35:05.688014 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.687929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59812)
0808 00:35:05.687969 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.687998 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:35:05.688139 19382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41057: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
I20260808 00:35:05.719316 19382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } attempt: 1
I20260808 00:35:05.719941 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.719756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59820)
0808 00:35:05.719847 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.719905 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 00:35:05.720402 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.720291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59822)
0808 00:35:05.720342 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.720373 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:35:05.720549 19382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41057: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
W20260808 00:35:05.741580 19492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:05.741807 19492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:05.741832 19492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:05.742987 19492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:05.743022 19492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:05.743036 19492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:05.743049 19492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:05.743062 19492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:05.744510 19492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:36565,127.31.210.253:37061,127.31.210.252:41057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab
--principal=kudu/127.31.210.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:41057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.210.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:05.744705 19492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:05.744884 19492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:35:05.746111 19482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } attempt: 2
I20260808 00:35:05.746562 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.746409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59834)
0808 00:35:05.746476 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.746523 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 00:35:05.746943 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.746838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59844)
0808 00:35:05.746886 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.746922 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:35:05.747077 19482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41057: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41057: connect: Connection refused (error 111)
W20260808 00:35:05.750224 19503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:05.750255 19499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:05.750224 19498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:05.750522 19492 server_base.cc:1034] running on GCE node
I20260808 00:35:05.750742 19492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:05.751017 19492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:05.752170 19492 hybrid_clock.cc:648] HybridClock initialized: now 1786149305752155 us; error 27 us; skew 500 ppm
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:05.754843 19492 init.cc:377] Logged in from keytab as kudu/127.31.210.252@KRBTEST.COM (short username kudu)
I20260808 00:35:05.755386 19492 webserver.cc:460] Webserver started at http://127.31.210.252:35495/ using document root <none> and password file <none>
I20260808 00:35:05.755618 19492 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:05.755684 19492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:05.755831 19492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:05.757032 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data/instance:
uuid: "967e118d761947f2813a4a71fa3a0616"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "0844a521a06ab3d71869e812784459ae"
server_key_iv: "5fdd9eadce5067f81ba62ef9ad35229c"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.757409 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/wal/instance:
uuid: "967e118d761947f2813a4a71fa3a0616"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "0844a521a06ab3d71869e812784459ae"
server_key_iv: "5fdd9eadce5067f81ba62ef9ad35229c"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.758775 19492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:35:05.759645 19514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:05.759819 19492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:35:05.759922 19492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/wal
uuid: "967e118d761947f2813a4a71fa3a0616"
format_stamp: "Formatted at 2026-08-08 00:35:05 on dist-test-slave-44d2"
server_key: "0844a521a06ab3d71869e812784459ae"
server_key_iv: "5fdd9eadce5067f81ba62ef9ad35229c"
server_key_version: "encryptionkey@0"
I20260808 00:35:05.759986 19492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:05.776592 19492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:05.778787 19492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:05.779045 19492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:05.783671 19492 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.252:41057
I20260808 00:35:05.783711 19585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.252:41057 every 8 connection(s)
I20260808 00:35:05.784112 19492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
I20260808 00:35:05.785177 19586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:05.787142 19586 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:35:05.789943 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 19492
I20260808 00:35:05.790081 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149138237870-32587-0/minicluster-data/master-2/wal/instance
I20260808 00:35:05.790262 32587 external_mini_cluster.cc:1428] Setting key 226e8f0b8a4099fd3243c238526e7384
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:05.795709 19382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } attempt: 2
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:35:05.798092 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.787964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.31.210.254:36565)
0808 00:35:05.788032 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.788034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.788275 (+ 241us) server_negotiation.cc:407] Connection header received
0808 00:35:05.788874 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.788875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.788976 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.789059 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.789745 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.790429 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.791545 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.791926 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.791991 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.794770 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.794771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.794772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.794780 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.796545 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.797027 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.797028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.797029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.797088 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.797380 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.797381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.797381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.797533 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.797608 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.797898 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.797993 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:35:05.798282 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.787927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:36565 (local address 127.0.0.1:44840)
0808 00:35:05.788149 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.788188 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.788760 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.789145 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.789149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.789435 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.789641 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.789645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.790541 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.790542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.791440 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.791442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.792010 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.792011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.792158 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.792803 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.792813 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.794656 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.796640 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.796649 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.796652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.796930 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.797157 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.797158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.797159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.797299 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.797599 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.797606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.797746 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.797857 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.798031 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":132,"threads_started":1}
I20260808 00:35:05.799041 19586 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:35:05.801326 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.790688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.31.210.252:41057)
0808 00:35:05.791000 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.791004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.791021 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:35:05.791040 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.791041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.791058 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.791144 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.791864 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.792968 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.794026 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.794373 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.794432 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.797234 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.797240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.797240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.797246 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.799788 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.800213 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.800217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.800217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.800342 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.800568 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.800569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.800569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.800772 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.800932 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.801048 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.801123 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":100,"threads_started":1}
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:35:05.804299 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.795975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59866)
0808 00:35:05.796083 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.796094 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.796140 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.796374 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.796376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.796594 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.796639 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.796640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.797562 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.797563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.798116 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.798118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.798795 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.798797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.798858 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.799374 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.799380 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.800962 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.803332 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.803333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.803333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.803518 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.803706 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.803706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.803707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.803770 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.804004 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.804005 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.804048 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.804115 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.804203 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 00:35:05.804905 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.796098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59866 (local address 127.31.210.252:41057)
0808 00:35:05.796243 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.796245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.796251 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:35:05.796269 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.796270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.796279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.796352 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.796721 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.797442 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.798409 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.798696 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.799011 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.801057 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.801058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.801059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.801065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.803226 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.803584 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.803584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.803584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.803630 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.803828 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.803829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.803829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.803935 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.804010 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.804700 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.804778 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:05.807291 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.799289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:37061 (local address 127.0.0.1:48756)
0808 00:35:05.799336 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.799348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.799399 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.799537 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.799538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.799720 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.799753 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.799755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.800674 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.800675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.801409 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.801411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.801759 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.801761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.801820 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.802695 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.802699 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.804002 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.806142 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.806143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.806143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.806374 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.806627 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.806628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.806629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.806700 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.806979 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.806980 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.807025 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.807107 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.807203 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:35:05.807312 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.799347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48756 (local address 127.31.210.253:37061)
0808 00:35:05.799390 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.799392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.799401 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:35:05.799457 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.799458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.799469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.799534 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.799835 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.800592 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.801484 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.801688 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.801763 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.804101 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.804102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.804102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.804110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.806056 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.806471 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.806473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.806474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.806555 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.806786 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.806788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.806788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.806901 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.806982 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.807146 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.807221 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:35:05.807399 19382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: Bootstrap starting.
I20260808 00:35:05.807871 19586 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:35:05.808247 19382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:05.808650 19382 log.cc:824] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:05.809947 19382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: No bootstrap required, opened a new log
I20260808 00:35:05.811215 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.802581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.31.210.254:36565)
0808 00:35:05.802623 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.802625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.802633 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:35:05.802657 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.802658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.802668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.802727 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.803335 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.804034 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.805013 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.805258 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.805338 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.808342 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.808343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.808344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.808351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.810064 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.810430 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.810431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.810431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.810498 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.810697 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.810698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.810698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.810845 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.811022 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.811100 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.811161 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:35:05.812471 19382 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:05.812690 19382 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:05.812825 19382 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ee15ae82eb4dbfa40329d6d476cb40, State: Initialized, Role: FOLLOWER
I20260808 00:35:05.813099 19382 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:05.813481 19382 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:05.813712 19594 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:05.815781 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.808202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59882)
0808 00:35:05.808243 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.808253 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.808297 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.808410 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.808411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.808587 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.808612 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.808613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.809399 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.809400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.810131 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.810134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.810615 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.810617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.810672 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.811295 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.811300 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.812728 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.814683 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.814683 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.814684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.814911 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.815133 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.815134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.815134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.815192 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.815452 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.815452 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.815494 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.815565 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.815670 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:35:05.815838 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.808229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59882 (local address 127.31.210.252:41057)
0808 00:35:05.808260 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.808261 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.808283 (+ 22us) server_negotiation.cc:407] Connection header received
0808 00:35:05.808352 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.808352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.808361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.808409 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.808684 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.809325 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.810207 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.810533 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.810602 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.812833 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.812834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.812834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.812840 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.814583 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.814989 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.814990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.814990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.815061 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.815242 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.815243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.815244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.815378 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.815461 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.815590 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.815721 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":16}
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:35:05.815287 19594 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:35:05.817523 19586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: Bootstrap starting.
I20260808 00:35:05.818351 19586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:05.818717 19586 log.cc:824] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:05.819706 19586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: No bootstrap required, opened a new log
W20260808 00:35:05.815685 19610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:35:05.820675 19610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:35:05.818225 19276 master_runner.cc:186] Error getting master registration for 127.31.210.253:37061: OK, instance_id { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" instance_seqno: 1786149305603373 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:35:05.821870 19586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:05.822052 19276 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:35:05.822084 19586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:05.822170 19586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 967e118d761947f2813a4a71fa3a0616, State: Initialized, Role: FOLLOWER
I20260808 00:35:05.822309 19586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:05.822615 19613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:05.822678 19586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:05.822751 19613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:35:05.823372 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.811842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.31.210.253:37061)
0808 00:35:05.811882 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.811884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.811892 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:35:05.811949 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.811950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.811958 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.812014 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.812482 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.813177 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.814118 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.814398 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.814474 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.817204 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.817205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.817205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.817214 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.822076 (+ 4862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.822524 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.822526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.822526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.822589 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.822791 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.822792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.822793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.822935 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.823102 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.823188 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.823276 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:35:05.825304 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 00:35:05.825771 19626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:35:05.825827 19626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:35:05.826373 19492 master_runner.cc:186] Error getting master registration for 127.31.210.253:37061: OK, instance_id { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" instance_seqno: 1786149305603373 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:35:05.828805 19492 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:35:05.837237 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.827386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.31.210.253:37061)
0808 00:35:05.827468 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.827470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.829670 (+ 2200us) server_negotiation.cc:407] Connection header received
0808 00:35:05.829715 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.829716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.829727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.829778 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.830489 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.831220 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.832316 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.832690 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.832733 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.834309 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.834310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.834310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.834317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.835900 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.836311 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.836312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.836313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.836367 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.836601 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.836602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.836602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.836732 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.836828 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.837056 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.837137 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 00:35:05.837723 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.827348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.31.210.254:36565)
0808 00:35:05.827437 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.827439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.828656 (+ 1217us) server_negotiation.cc:407] Connection header received
0808 00:35:05.828759 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.828760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.828771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.828802 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.829733 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.830483 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.831545 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.831837 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.831880 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.833992 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.833994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.833994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.834002 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.836251 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.836787 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.836788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.836788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.836849 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.837071 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.837072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.837073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.837202 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.837280 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.837549 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.837630 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 00:35:05.838223 19482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } attempt: 3
Aug 08 00:35:05 dist-test-slave-44d2 krb5kdc[19258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149305, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:35:05.845209 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.827624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.31.210.252:41057)
0808 00:35:05.827659 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.827661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.837762 (+ 10101us) server_negotiation.cc:407] Connection header received
0808 00:35:05.837863 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.837864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.837875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.837907 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.838700 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.839440 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.840430 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.840752 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.840792 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.842171 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.842173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.842173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.842181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.843753 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.844131 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.844132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.844132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.844260 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.844544 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.844545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.844546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.844726 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.844815 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.845036 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.845117 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:35:05.847102 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.838497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41057 (local address 127.0.0.1:59912)
0808 00:35:05.838568 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:05.838578 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:05.838617 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:05.838845 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:05.838846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:05.839021 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:05.839059 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.839061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.840005 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.840006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.840771 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:05.840773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.841655 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.841656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:05.841711 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.842130 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:05.842135 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:05.844170 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:05.846095 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.846096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.846096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.846303 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.846493 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:05.846494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:05.846494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:05.846547 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:05.846799 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:05.846800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:05.846840 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 00:35:05.846899 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:35:05.847001 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 00:35:05.847892 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:05.838686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.31.210.252:41057)
0808 00:35:05.838724 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:05.838726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:05.838732 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:35:05.838751 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:05.838752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:05.838761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:05.838807 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:05.839147 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.839890 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.840866 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:05.841538 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:05.841602 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:05.844266 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:05.844267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:05.844267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:05.844274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:05.845998 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.846379 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.846379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.846380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.846429 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:05.846609 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:05.846609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:05.846610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:05.846723 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:05.846776 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:05.847725 (+ 949us) server_negotiation.cc:299] Negotiation successful
0808 00:35:05.847802 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:35:05.849507 19482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238: Bootstrap starting.
I20260808 00:35:05.850267 19482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:05.850590 19482 log.cc:824] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:05.851545 19482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238: No bootstrap required, opened a new log
I20260808 00:35:05.853683 19482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:05.853834 19482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:05.853871 19482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc672ee843d48d3a2ef2decd3af6238, State: Initialized, Role: FOLLOWER
I20260808 00:35:05.854053 19482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:05.854341 19642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:05.854393 19482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:05.854475 19642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:35:05.856406 19656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:35:05.856595 19656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:35:05.858341 19392 master_runner.cc:263] No leader master found from master 8fc672ee843d48d3a2ef2decd3af6238
I20260808 00:35:05.858390 19392 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 00:35:05.943778 19313 tablet.cc:2369] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:35:06.029729 19521 tablet.cc:2369] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:35:06.069077 19642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:06.069217 19642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:06.069475 19642 leader_election.cc:288] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7ee15ae82eb4dbfa40329d6d476cb40 (127.31.210.254:36565), 967e118d761947f2813a4a71fa3a0616 (127.31.210.252:41057)
I20260808 00:35:06.069824 19552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fc672ee843d48d3a2ef2decd3af6238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "967e118d761947f2813a4a71fa3a0616" is_pre_election: true
I20260808 00:35:06.069869 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fc672ee843d48d3a2ef2decd3af6238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" is_pre_election: true
I20260808 00:35:06.070029 19552 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fc672ee843d48d3a2ef2decd3af6238 in term 0.
I20260808 00:35:06.070101 19344 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fc672ee843d48d3a2ef2decd3af6238 in term 0.
I20260808 00:35:06.070344 19416 leader_election.cc:302] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fc672ee843d48d3a2ef2decd3af6238, 967e118d761947f2813a4a71fa3a0616; no voters:
I20260808 00:35:06.070568 19642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:35:06.070641 19642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:35:06.070664 19642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:06.071532 19642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:06.071705 19642 leader_election.cc:288] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [CANDIDATE]: Term 1 election: Requested vote from peers a7ee15ae82eb4dbfa40329d6d476cb40 (127.31.210.254:36565), 967e118d761947f2813a4a71fa3a0616 (127.31.210.252:41057)
I20260808 00:35:06.071899 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fc672ee843d48d3a2ef2decd3af6238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40"
I20260808 00:35:06.071888 19552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fc672ee843d48d3a2ef2decd3af6238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "967e118d761947f2813a4a71fa3a0616"
I20260808 00:35:06.071982 19344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:06.072010 19552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:06.072842 19344 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fc672ee843d48d3a2ef2decd3af6238 in term 1.
I20260808 00:35:06.073047 19413 leader_election.cc:302] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fc672ee843d48d3a2ef2decd3af6238, a7ee15ae82eb4dbfa40329d6d476cb40; no voters:
I20260808 00:35:06.073051 19552 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fc672ee843d48d3a2ef2decd3af6238 in term 1.
I20260808 00:35:06.073139 19642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:06.073313 19642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [term 1 LEADER]: Becoming Leader. State: Replica: 8fc672ee843d48d3a2ef2decd3af6238, State: Running, Role: LEADER
I20260808 00:35:06.073436 19642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } }
I20260808 00:35:06.073956 19662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fc672ee843d48d3a2ef2decd3af6238. Latest consensus state: current_term: 1 leader_uuid: "8fc672ee843d48d3a2ef2decd3af6238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:06.074069 19662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:06.074322 19664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:06.074496 19664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:06.075973 19344 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [term 1 FOLLOWER]: Refusing update from remote peer 8fc672ee843d48d3a2ef2decd3af6238: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:06.076107 19552 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [term 1 FOLLOWER]: Refusing update from remote peer 8fc672ee843d48d3a2ef2decd3af6238: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:06.076319 19662 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:06.076500 19642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [LEADER]: Connected to new peer: Peer: permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:06.077607 19594 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fc672ee843d48d3a2ef2decd3af6238. Latest consensus state: current_term: 1 leader_uuid: "8fc672ee843d48d3a2ef2decd3af6238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:06.077703 19594 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:35:06.077857 19613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fc672ee843d48d3a2ef2decd3af6238. Latest consensus state: current_term: 1 leader_uuid: "8fc672ee843d48d3a2ef2decd3af6238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:06.077948 19613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:35:06.078737 19642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fc672ee843d48d3a2ef2decd3af6238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:06.078823 19642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:06.078864 19642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fc672ee843d48d3a2ef2decd3af6238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:06.078902 19642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238 [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:06.079042 19594 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fc672ee843d48d3a2ef2decd3af6238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:06.079152 19594 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:35:06.079181 19664 catalog_manager.cc:1347] Generated new cluster ID: d787df30fccd414294a6357017a44164
I20260808 00:35:06.079247 19664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:35:06.080586 19613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fc672ee843d48d3a2ef2decd3af6238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ee15ae82eb4dbfa40329d6d476cb40" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36565 } } peers { permanent_uuid: "8fc672ee843d48d3a2ef2decd3af6238" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 37061 } } peers { permanent_uuid: "967e118d761947f2813a4a71fa3a0616" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41057 } } }
I20260808 00:35:06.080687 19613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:35:06.080809 19666 mvcc.cc:205] Tried to move back new op lower bound from 7316067557684342784 to 7316067557677318144. Current Snapshot: MvccSnapshot[applied={T|T < 7316067557684342784 or (T in {7316067557684342784})}]
I20260808 00:35:06.080931 19667 mvcc.cc:205] Tried to move back new op lower bound from 7316067557684342784 to 7316067557677318144. Current Snapshot: MvccSnapshot[applied={T|T < 7316067557684342784}]
I20260808 00:35:06.114198 19664 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:35:06.114998 19664 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:35:06.120824 19664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fc672ee843d48d3a2ef2decd3af6238: Generated new TSK 0
I20260808 00:35:06.121033 19664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:35:06.821614 19610 catalog_manager.cc:1259] Loaded cluster ID: d787df30fccd414294a6357017a44164
I20260808 00:35:06.821697 19610 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: loading cluster ID for follower catalog manager: success
I20260808 00:35:06.822525 19610 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: acquiring CA information for follower catalog manager: success
I20260808 00:35:06.822806 19610 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a7ee15ae82eb4dbfa40329d6d476cb40: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:35:06.826227 19626 catalog_manager.cc:1259] Loaded cluster ID: d787df30fccd414294a6357017a44164
I20260808 00:35:06.826308 19626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: loading cluster ID for follower catalog manager: success
I20260808 00:35:06.827226 19626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: acquiring CA information for follower catalog manager: success
I20260808 00:35:06.827528 19626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 967e118d761947f2813a4a71fa3a0616: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:35:07.176775 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 19276
I20260808 00:35:07.190326 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 19392
I20260808 00:35:07.201761 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 19492
2026-08-08T00:35:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3286 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:35:07 dist-test-slave-44d2 krb5kdc[19682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:35:07 dist-test-slave-44d2 krb5kdc[19682](info): set up 2 sockets
Aug 08 00:35:07 dist-test-slave-44d2 krb5kdc[19682](info): commencing operation
krb5kdc: starting...
W20260808 00:35:08.484354 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:35:08 dist-test-slave-44d2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:35:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:35:08.623848 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:33223
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:33223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:08.712356 19701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:08.712684 19701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:08.712709 19701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:08.713843 19701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:08.713882 19701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:08.713897 19701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:08.713910 19701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:08.713922 19701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:08.715344 19701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:33223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:33223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:08.715550 19701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:08.715725 19701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:08.721433 19714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:08.721536 19711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:08.721433 19712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:08.721746 19701 server_base.cc:1034] running on GCE node
I20260808 00:35:08.722034 19701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:08.722298 19701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:08.723454 19701 hybrid_clock.cc:648] HybridClock initialized: now 1786149308723441 us; error 31 us; skew 500 ppm
Aug 08 00:35:08 dist-test-slave-44d2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149308, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:08.730043 19701 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:08.730672 19701 webserver.cc:460] Webserver started at http://127.31.210.254:39417/ using document root <none> and password file <none>
I20260808 00:35:08.730952 19701 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:08.731026 19701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:08.731189 19701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:08.732420 19701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "8d1bca7419fa42b68ef0b2367fa88159"
format_stamp: "Formatted at 2026-08-08 00:35:08 on dist-test-slave-44d2"
server_key: "146dbc86df8e80a68c728fc94498e03a"
server_key_iv: "85d4d507f3a0f3a6a42d61d41270c466"
server_key_version: "encryptionkey@0"
I20260808 00:35:08.732872 19701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "8d1bca7419fa42b68ef0b2367fa88159"
format_stamp: "Formatted at 2026-08-08 00:35:08 on dist-test-slave-44d2"
server_key: "146dbc86df8e80a68c728fc94498e03a"
server_key_iv: "85d4d507f3a0f3a6a42d61d41270c466"
server_key_version: "encryptionkey@0"
I20260808 00:35:08.734459 19701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:35:08.735278 19722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:08.735507 19701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:08.735618 19701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "8d1bca7419fa42b68ef0b2367fa88159"
format_stamp: "Formatted at 2026-08-08 00:35:08 on dist-test-slave-44d2"
server_key: "146dbc86df8e80a68c728fc94498e03a"
server_key_iv: "85d4d507f3a0f3a6a42d61d41270c466"
server_key_version: "encryptionkey@0"
I20260808 00:35:08.735694 19701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:08.754057 19701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:08.756682 19701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:08.756852 19701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:08.761436 19701 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:33223
I20260808 00:35:08.761482 19803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:33223 every 8 connection(s)
I20260808 00:35:08.761855 19701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:08.762769 19804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:08.765241 19804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159: Bootstrap starting.
I20260808 00:35:08.765843 19804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:08.766160 19804 log.cc:824] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:08.766916 19804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159: No bootstrap required, opened a new log
I20260808 00:35:08.768544 19804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1bca7419fa42b68ef0b2367fa88159" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33223 } }
I20260808 00:35:08.768685 19804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:08.768707 19804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1bca7419fa42b68ef0b2367fa88159, State: Initialized, Role: FOLLOWER
I20260808 00:35:08.768873 19804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1bca7419fa42b68ef0b2367fa88159" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33223 } }
I20260808 00:35:08.768954 19804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:35:08.768992 19804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:35:08.769060 19804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:08.769613 19804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1bca7419fa42b68ef0b2367fa88159" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33223 } }
I20260808 00:35:08.769727 19804 leader_election.cc:302] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d1bca7419fa42b68ef0b2367fa88159; no voters:
I20260808 00:35:08.769910 19804 leader_election.cc:288] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:35:08.769959 19812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:08.770020 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 19701
I20260808 00:35:08.770134 19812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1bca7419fa42b68ef0b2367fa88159, State: Running, Role: LEADER
I20260808 00:35:08.770153 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:35:08.770228 19804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:08.770273 19812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1bca7419fa42b68ef0b2367fa88159" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33223 } }
I20260808 00:35:08.770407 32587 external_mini_cluster.cc:1428] Setting key 3e4796acf5a4aa8ca658a5e36eb2ca10
I20260808 00:35:08.770818 19813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d1bca7419fa42b68ef0b2367fa88159. Latest consensus state: current_term: 1 leader_uuid: "8d1bca7419fa42b68ef0b2367fa88159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1bca7419fa42b68ef0b2367fa88159" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33223 } } }
I20260808 00:35:08.771008 19813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:08.771494 19816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d1bca7419fa42b68ef0b2367fa88159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1bca7419fa42b68ef0b2367fa88159" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33223 } } }
I20260808 00:35:08.771647 19816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159 [sys.catalog]: This master's current role is: LEADER
W20260808 00:35:08.772187 19837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:35:08.772286 19837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:35:08.772377 19826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:08.772568 19826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:08.774144 19826 catalog_manager.cc:1347] Generated new cluster ID: 3e0caa888696433580e9bfc34749f24b
I20260808 00:35:08.774200 19826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:35:08 dist-test-slave-44d2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:08.781759 19826 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:35:08.782253 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:08.771094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.31.210.254:33223)
0808 00:35:08.771638 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:08.771648 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:08.771962 (+ 314us) server_negotiation.cc:407] Connection header received
0808 00:35:08.772088 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:08.772093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:08.772138 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:08.772274 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:08.773073 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:08.774481 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:08.775231 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:08.775669 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:08.775782 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:08.778526 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:08.778530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:08.778531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:08.778542 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:08.780385 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:08.780836 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:08.780839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:08.780840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:08.780954 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:08.781187 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:08.781187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:08.781188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:08.781521 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:08.781735 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:08.781867 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 00:35:08.782021 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":92,"threads_started":1}
I20260808 00:35:08.782521 19826 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:35:08.783322 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:35:08.789295 19826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d1bca7419fa42b68ef0b2367fa88159: Generated new TSK 0
I20260808 00:35:08.789498 19826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:35:08.837790 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 19701
2026-08-08T00:35:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1641 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:35:08 dist-test-slave-44d2 krb5kdc[19845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:35:08 dist-test-slave-44d2 krb5kdc[19845](info): set up 2 sockets
Aug 08 00:35:08 dist-test-slave-44d2 krb5kdc[19845](info): commencing operation
krb5kdc: starting...
W20260808 00:35:10.159173 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:35:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:35:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:35:10.293778 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44411
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:44411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:10.377274 19867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:10.377490 19867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:10.377514 19867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:10.378703 19867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:10.378754 19867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:10.378769 19867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:10.378782 19867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:10.378794 19867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:10.380321 19867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:44411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:10.380601 19867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:10.380827 19867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:10.386188 19883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:10.386229 19875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:10.386188 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:10.386417 19867 server_base.cc:1034] running on GCE node
I20260808 00:35:10.386708 19867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:10.386996 19867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:10.388139 19867 hybrid_clock.cc:648] HybridClock initialized: now 1786149310388129 us; error 30 us; skew 500 ppm
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:10.390717 19867 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:10.391182 19867 webserver.cc:460] Webserver started at http://127.31.210.254:45825/ using document root <none> and password file <none>
I20260808 00:35:10.391407 19867 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:10.391484 19867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:10.391626 19867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:10.392768 19867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "19b6d95363a4485fb202574a10fd800a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "a9171d942bc3ae82c2066314a1015906"
server_key_iv: "466a64cdb3adc7010517e108c2c3cab6"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.393139 19867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "19b6d95363a4485fb202574a10fd800a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "a9171d942bc3ae82c2066314a1015906"
server_key_iv: "466a64cdb3adc7010517e108c2c3cab6"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.394520 19867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:10.395447 19893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.395649 19867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:10.395752 19867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "19b6d95363a4485fb202574a10fd800a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "a9171d942bc3ae82c2066314a1015906"
server_key_iv: "466a64cdb3adc7010517e108c2c3cab6"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.395823 19867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:10.422173 19867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:10.424731 19867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:10.424928 19867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:10.429508 19867 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:44411
I20260808 00:35:10.429543 19966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:44411 every 8 connection(s)
I20260808 00:35:10.429960 19867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:10.431154 19967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:10.433645 19967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Bootstrap starting.
I20260808 00:35:10.434226 19967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:10.434497 19967 log.cc:824] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:10.435223 19967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: No bootstrap required, opened a new log
I20260808 00:35:10.437842 19967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:10.438035 19967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:10.438072 19967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b6d95363a4485fb202574a10fd800a, State: Initialized, Role: FOLLOWER
I20260808 00:35:10.438227 19967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:10.438315 19967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:35:10.438360 19967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:35:10.438427 19967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:10.439515 19967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:10.439613 19967 leader_election.cc:302] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19b6d95363a4485fb202574a10fd800a; no voters:
I20260808 00:35:10.439759 19967 leader_election.cc:288] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:35:10.439814 19972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:10.439924 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 19867
I20260808 00:35:10.440024 19972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 LEADER]: Becoming Leader. State: Replica: 19b6d95363a4485fb202574a10fd800a, State: Running, Role: LEADER
I20260808 00:35:10.440052 19967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:10.440091 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:35:10.440120 19972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:10.440325 32587 external_mini_cluster.cc:1428] Setting key 833d37be01e984a8e82c493e8b2b732c
I20260808 00:35:10.440526 19973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19b6d95363a4485fb202574a10fd800a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } } }
I20260808 00:35:10.440603 19973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:10.440471 19972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19b6d95363a4485fb202574a10fd800a. Latest consensus state: current_term: 1 leader_uuid: "19b6d95363a4485fb202574a10fd800a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } } }
I20260808 00:35:10.440830 19972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:10.441366 19981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:10.441898 19981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:10.443567 19981 catalog_manager.cc:1347] Generated new cluster ID: c6588ffbb5824493a307f6bfb88728d1
I20260808 00:35:10.443634 19981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:10.453164 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:10.440997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.31.210.254:44411)
0808 00:35:10.441195 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:10.441208 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:10.441253 (+ 45us) server_negotiation.cc:407] Connection header received
0808 00:35:10.441464 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:10.441473 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:10.441546 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:10.441918 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:10.442844 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.444207 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.445393 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.446039 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.446148 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:10.449164 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:10.449169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:10.449170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:10.449183 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:10.451323 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.451741 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.451744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.451744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.451866 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.452097 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.452098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.452098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.452429 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:10.452675 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:10.452756 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 00:35:10.452913 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260808 00:35:10.465023 19981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:35:10.465653 19981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:35:10.473002 19981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Generated new TSK 0
I20260808 00:35:10.473212 19981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:35:10.512883 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44411
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:10.613250 19999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:10.613498 19999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:10.613535 19999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:10.614735 19999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:10.614804 19999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:10.614849 19999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:35:10.616345 19999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:10.616653 19999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:10.616894 19999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:10.622215 20005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:10.622215 20009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:10.622275 20007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:10.622581 19999 server_base.cc:1034] running on GCE node
I20260808 00:35:10.622787 19999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:10.623032 19999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:10.624189 19999 hybrid_clock.cc:648] HybridClock initialized: now 1786149310624175 us; error 28 us; skew 500 ppm
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:10.627106 19999 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:35:10.627554 19999 webserver.cc:460] Webserver started at http://127.31.210.193:45155/ using document root <none> and password file <none>
I20260808 00:35:10.627758 19999 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:10.627804 19999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:10.627938 19999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:10.629084 19999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "da51014a186241b4b98612265fc560a9"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "21719a0788395cab112facbb0a1623a9"
server_key_iv: "6cdb0483e10c4ac567656602f524dfb6"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.629429 19999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "da51014a186241b4b98612265fc560a9"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "21719a0788395cab112facbb0a1623a9"
server_key_iv: "6cdb0483e10c4ac567656602f524dfb6"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.631028 19999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:35:10.631834 20018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.632027 19999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:10.632143 19999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "da51014a186241b4b98612265fc560a9"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "21719a0788395cab112facbb0a1623a9"
server_key_iv: "6cdb0483e10c4ac567656602f524dfb6"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.632230 19999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:10.644814 19999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:10.648043 19999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:10.648217 19999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:10.648512 19999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:10.648909 19999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:10.648959 19999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.649005 19999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:10.649037 19999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.656586 19999 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:42683
I20260808 00:35:10.656641 20189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:42683 every 8 connection(s)
I20260808 00:35:10.656994 19999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:35:10.660382 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 19999
I20260808 00:35:10.660521 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:35:10.660723 32587 external_mini_cluster.cc:1428] Setting key 0b5bb02da21376813b058691203c0983
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:10.667160 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:10.657759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:52275 (local address 127.31.210.254:44411)
0808 00:35:10.657817 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:10.657819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:10.658099 (+ 280us) server_negotiation.cc:407] Connection header received
0808 00:35:10.658644 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:10.658645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:10.658663 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:10.658715 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:10.659690 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.660256 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.661578 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.661907 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.661986 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:10.664251 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:10.664253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:10.664254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:10.664262 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:10.665713 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.666119 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.666121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.666121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.666186 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.666440 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.666441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.666441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.666594 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:10.666674 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:10.666974 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 00:35:10.667069 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:35:10.667388 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:10.657747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:52275)
0808 00:35:10.657917 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:10.657955 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:10.658531 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:10.658786 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:10.658790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:10.659382 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:10.659585 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:10.659590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.660360 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.660361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:10.661497 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:10.661500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.661992 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.661993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:10.662099 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:10.662665 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:10.662674 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:10.664155 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:10.665798 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:10.665805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:10.665807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:10.666042 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:10.666257 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:10.666258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:10.666258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:10.666368 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:10.666684 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:10.666692 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:10.666831 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 00:35:10.666935 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:35:10.667107 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260808 00:35:10.667935 20192 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44411
I20260808 00:35:10.668047 20192 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:10.668229 20192 heartbeater.cc:511] Master 127.31.210.254:44411 requested a full tablet report, sending...
I20260808 00:35:10.668866 19921 ts_manager.cc:194] Registered new tserver with Master: da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:10.669831 19921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:52275
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:35:10.691700 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44411
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:10.776734 20202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:10.776965 20202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:10.777004 20202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:10.778141 20202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:10.778180 20202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:10.778206 20202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:35:10.779588 20202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:10.779769 20202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:10.779944 20202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:10.785252 20210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:10.785260 20218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:10.785284 20213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:10.785495 20202 server_base.cc:1034] running on GCE node
I20260808 00:35:10.785821 20202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:10.786032 20202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:10.787179 20202 hybrid_clock.cc:648] HybridClock initialized: now 1786149310787162 us; error 32 us; skew 500 ppm
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:10.789829 20202 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:35:10.790304 20202 webserver.cc:460] Webserver started at http://127.31.210.194:38839/ using document root <none> and password file <none>
I20260808 00:35:10.790555 20202 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:10.790623 20202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:10.790774 20202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:10.791838 20202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "f432c21575f84cfc9c5cac352c8f482b"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "0ca6aa7ef212ecaa015c1082f3c696f3"
server_key_iv: "db8d6b165a96667973ab035924450a7e"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.792228 20202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "f432c21575f84cfc9c5cac352c8f482b"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "0ca6aa7ef212ecaa015c1082f3c696f3"
server_key_iv: "db8d6b165a96667973ab035924450a7e"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.793687 20202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:35:10.794468 20225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.794638 20202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:35:10.794728 20202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "f432c21575f84cfc9c5cac352c8f482b"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "0ca6aa7ef212ecaa015c1082f3c696f3"
server_key_iv: "db8d6b165a96667973ab035924450a7e"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.794809 20202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:10.810024 20202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:10.812171 20202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:10.812357 20202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:10.812649 20202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:10.813014 20202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:10.813053 20202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.813077 20202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:10.813090 20202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.819088 20202 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:46145
I20260808 00:35:10.819130 20380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:46145 every 8 connection(s)
I20260808 00:35:10.819474 20202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:10.827847 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 20202
I20260808 00:35:10.828027 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:35:10.828270 32587 external_mini_cluster.cc:1428] Setting key 268c8054d838c6802b763aa8d9ecbcd9
I20260808 00:35:10.829988 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:10.820299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:39855 (local address 127.31.210.254:44411)
0808 00:35:10.820384 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:10.820386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:10.820588 (+ 202us) server_negotiation.cc:407] Connection header received
0808 00:35:10.821123 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:10.821125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:10.821137 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:10.821187 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:10.822300 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.822809 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.823915 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.824112 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.824170 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:10.826539 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:10.826541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:10.826542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:10.826548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:10.828141 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.828759 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.828761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.828761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.828840 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.829190 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.829191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.829191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.829320 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:10.829391 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:10.829789 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 00:35:10.829886 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:35:10.830207 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:10.820245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:39855)
0808 00:35:10.820409 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:10.820457 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:10.821030 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:10.821257 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:10.821260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:10.821994 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:10.822210 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:10.822215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.822894 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.822895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:10.823828 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:10.823832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.824193 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.824194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:10.824307 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:10.824960 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:10.824972 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:10.826410 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:10.828236 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:10.828245 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:10.828247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:10.828642 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:10.828932 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:10.828933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:10.828934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:10.829057 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:10.829388 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:10.829396 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:10.829624 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 00:35:10.829737 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 00:35:10.829916 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260808 00:35:10.831552 20381 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44411
I20260808 00:35:10.831650 20381 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:10.831856 20381 heartbeater.cc:511] Master 127.31.210.254:44411 requested a full tablet report, sending...
I20260808 00:35:10.832329 19921 ts_manager.cc:194] Registered new tserver with Master: f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
I20260808 00:35:10.832899 19921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:39855
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:35:10.857265 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44411
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:10.943668 20391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:10.943864 20391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:10.943889 20391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:10.945041 20391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:10.945083 20391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:10.945109 20391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:35:10.946496 20391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:10.946689 20391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:10.946874 20391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:10.952107 20403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:10.952028 20402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:10.952150 20391 server_base.cc:1034] running on GCE node
W20260808 00:35:10.952039 20406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:10.952595 20391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:10.952867 20391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:10.954057 20391 hybrid_clock.cc:648] HybridClock initialized: now 1786149310954038 us; error 31 us; skew 500 ppm
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:10.956821 20391 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:35:10.957231 20391 webserver.cc:460] Webserver started at http://127.31.210.195:36683/ using document root <none> and password file <none>
I20260808 00:35:10.957439 20391 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:10.957484 20391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:10.957598 20391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:10.958632 20391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "de4dce29fc0e4ee38e23d049f48eb14a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "db89474d913664f5a1cd9b587969da83"
server_key_iv: "98618e7367a360407f73bf9d69604727"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.958947 20391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "de4dce29fc0e4ee38e23d049f48eb14a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "db89474d913664f5a1cd9b587969da83"
server_key_iv: "98618e7367a360407f73bf9d69604727"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.960250 20391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:35:10.961082 20416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.961277 20391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:35:10.961385 20391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "de4dce29fc0e4ee38e23d049f48eb14a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "db89474d913664f5a1cd9b587969da83"
server_key_iv: "98618e7367a360407f73bf9d69604727"
server_key_version: "encryptionkey@0"
I20260808 00:35:10.961460 20391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:10.976897 20391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:10.978889 20391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:10.979063 20391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:10.979302 20391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:10.979617 20391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:10.979669 20391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.979712 20391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:10.979748 20391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:10.986164 20391 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:34309
I20260808 00:35:10.986220 20576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:34309 every 8 connection(s)
I20260808 00:35:10.986593 20391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:35:10.993285 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 20391
Aug 08 00:35:10 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149310, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:10.993496 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:35:10.993680 32587 external_mini_cluster.cc:1428] Setting key f1a36d67bb1c4edf8be7b1725343f0a9
I20260808 00:35:10.996821 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:10.987589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:40089 (local address 127.31.210.254:44411)
0808 00:35:10.987651 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:10.987655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:10.987889 (+ 234us) server_negotiation.cc:407] Connection header received
0808 00:35:10.988382 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:10.988383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:10.988396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:10.988466 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:10.989570 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.990193 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.991236 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.991427 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.991514 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:10.994008 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:10.994010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:10.994011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:10.994018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:10.995604 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.996000 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.996001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.996001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.996055 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:10.996262 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:10.996262 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:10.996263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:10.996363 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:10.996413 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:10.996696 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 00:35:10.996757 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:35:10.997064 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:10.987576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:40089)
0808 00:35:10.987758 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:10.987795 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:10.988270 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:10.988584 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:10.988588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:10.989214 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:10.989416 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:10.989421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.990293 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.990294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:10.991131 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:10.991133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:10.991527 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:10.991528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:10.991640 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:10.992339 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:10.992348 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:10.993872 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:10.995683 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:10.995689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:10.995691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:10.995904 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:10.996111 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:10.996112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:10.996112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:10.996210 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:10.996413 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:10.996417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:10.996597 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 00:35:10.996683 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 00:35:10.996808 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260808 00:35:10.997514 20578 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44411
I20260808 00:35:10.997629 20578 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:10.997864 20578 heartbeater.cc:511] Master 127.31.210.254:44411 requested a full tablet report, sending...
I20260808 00:35:10.998221 19921 ts_manager.cc:194] Registered new tserver with Master: de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309)
I20260808 00:35:10.998646 19921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:40089
I20260808 00:35:11.005656 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:35:11.013631 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.007072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.31.210.254:44411)
0808 00:35:11.007133 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.007135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.007305 (+ 170us) server_negotiation.cc:407] Connection header received
0808 00:35:11.007333 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.007334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.007346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.007405 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:11.008055 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.008657 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.009499 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.009690 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.009757 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.011173 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:11.011174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:11.011175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:11.011182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:11.012661 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.012993 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.012994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.012994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.013048 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.013176 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.013177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.013177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.013318 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:11.013393 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.013506 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.013566 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:35:11.015144 19921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:35:11.016760 19921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:35:11.023597 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.019207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.31.210.193:42683)
0808 00:35:11.019743 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.019748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.019760 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:35:11.019800 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.019802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.019819 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.019891 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.020100 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.021000 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.022384 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.023151 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.023238 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.023294 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.023454 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.023508 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":100,"threads_started":1}
I20260808 00:35:11.023624 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.019080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:42683 (local address 127.0.0.1:55524)
0808 00:35:11.019526 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.019534 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.019588 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.019981 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.019981 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.019983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.020025 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.020026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.021098 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.021099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.022275 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.022278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.023247 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.023248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.023336 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.023351 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.023426 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.023490 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":41,"thread_start_us":85,"threads_started":1}
I20260808 00:35:11.023824 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.019542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.31.210.195:34309)
0808 00:35:11.019750 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.019755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.020216 (+ 461us) server_negotiation.cc:407] Connection header received
0808 00:35:11.020263 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.020265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.020281 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.020364 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.020517 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.021601 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.022743 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.023267 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.023339 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.023392 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.023667 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.023721 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
I20260808 00:35:11.023857 20595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.019290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34309 (local address 127.0.0.1:52604)
0808 00:35:11.020137 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.020147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.020196 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.020373 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.020374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.020376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.020411 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.020412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.021710 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.021711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.022664 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.022666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.023427 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.023427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.023481 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.023484 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.023532 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.023574 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":69,"threads_started":1}
I20260808 00:35:11.024047 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.019207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.31.210.194:46145)
0808 00:35:11.019411 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.019416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.019433 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:35:11.019488 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.019490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.019508 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.019686 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.019952 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.021022 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.022450 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.023173 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.023238 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.023294 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.023815 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.023862 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":96,"threads_started":1}
I20260808 00:35:11.023646 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.019080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:46145 (local address 127.0.0.1:50678)
0808 00:35:11.019258 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.019298 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.019366 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.019793 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.019796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.019799 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.019867 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.019869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.021137 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.021138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.022366 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.022368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.023332 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.023333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.023427 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.023432 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.023496 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.023536 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":25,"thread_start_us":69,"threads_started":1}
I20260808 00:35:11.024573 20479 tablet_service.cc:1467] Processing CreateTablet for tablet cc1465cda2db42dfbac3ef1ee16a1623 (DEFAULT_TABLE table=test-table [id=02e30044cf2f4de5baf3968c7dae6506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:11.024631 20099 tablet_service.cc:1467] Processing CreateTablet for tablet cc1465cda2db42dfbac3ef1ee16a1623 (DEFAULT_TABLE table=test-table [id=02e30044cf2f4de5baf3968c7dae6506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:11.024953 20099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc1465cda2db42dfbac3ef1ee16a1623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:11.024953 20479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc1465cda2db42dfbac3ef1ee16a1623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:11.028398 20601 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Bootstrap starting.
I20260808 00:35:11.028929 20292 tablet_service.cc:1467] Processing CreateTablet for tablet cc1465cda2db42dfbac3ef1ee16a1623 (DEFAULT_TABLE table=test-table [id=02e30044cf2f4de5baf3968c7dae6506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:11.029059 20601 tablet_bootstrap.cc:654] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:11.029263 20292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc1465cda2db42dfbac3ef1ee16a1623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:11.029627 20601 log.cc:824] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:11.029947 20603 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Bootstrap starting.
I20260808 00:35:11.030457 20601 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: No bootstrap required, opened a new log
I20260808 00:35:11.030543 20601 ts_tablet_manager.cc:1397] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:11.030651 20603 tablet_bootstrap.cc:654] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:11.031176 20603 log.cc:824] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:11.031857 20603 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: No bootstrap required, opened a new log
I20260808 00:35:11.031941 20603 ts_tablet_manager.cc:1397] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:11.032651 20601 raft_consensus.cc:348] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.032793 20608 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Bootstrap starting.
I20260808 00:35:11.032843 20601 raft_consensus.cc:374] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:11.032927 20601 raft_consensus.cc:729] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4dce29fc0e4ee38e23d049f48eb14a, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.033077 20601 consensus_queue.cc:260] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.033380 20601 ts_tablet_manager.cc:1428] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 00:35:11.033455 20608 tablet_bootstrap.cc:654] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:11.033530 20578 heartbeater.cc:503] Master 127.31.210.254:44411 was elected leader, sending a full tablet report...
I20260808 00:35:11.033811 20603 raft_consensus.cc:348] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.033974 20603 raft_consensus.cc:374] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:11.034016 20603 raft_consensus.cc:729] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da51014a186241b4b98612265fc560a9, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.034111 20608 log.cc:824] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:11.034127 20603 consensus_queue.cc:260] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.034415 20192 heartbeater.cc:503] Master 127.31.210.254:44411 was elected leader, sending a full tablet report...
I20260808 00:35:11.034403 20603 ts_tablet_manager.cc:1428] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:35:11.034905 20608 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: No bootstrap required, opened a new log
I20260808 00:35:11.034987 20608 ts_tablet_manager.cc:1397] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:11.036664 20608 raft_consensus.cc:348] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.036801 20608 raft_consensus.cc:374] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:11.036845 20608 raft_consensus.cc:729] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f432c21575f84cfc9c5cac352c8f482b, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.036958 20608 consensus_queue.cc:260] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.037221 20608 ts_tablet_manager.cc:1428] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:35:11.037261 20381 heartbeater.cc:503] Master 127.31.210.254:44411 was elected leader, sending a full tablet report...
I20260808 00:35:11.054484 20609 raft_consensus.cc:482] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:11.054777 20609 raft_consensus.cc:504] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.055260 20609 leader_election.cc:288] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145), da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:11.059319 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.055534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:38671 (local address 127.31.210.193:42683)
0808 00:35:11.055585 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.055587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.055709 (+ 122us) server_negotiation.cc:407] Connection header received
0808 00:35:11.055760 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.055761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.055771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.055826 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.056071 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.056840 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.058312 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.058914 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.058994 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.059003 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.059204 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.059246 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:35:11.059319 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.055514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:50227 (local address 127.31.210.194:46145)
0808 00:35:11.055563 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.055566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.055634 (+ 68us) server_negotiation.cc:407] Connection header received
0808 00:35:11.055713 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.055714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.055727 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.055793 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.056035 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.056840 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.058312 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.058914 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.058994 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.059003 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.059205 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.059246 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:35:11.059332 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.055429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:46145 (local address 127.31.210.195:50227)
0808 00:35:11.055538 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.055553 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.055622 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.055829 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.055830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.055832 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.055952 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.055954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.056953 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.056954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.058213 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.058215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.059025 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.059027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.059120 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.059122 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.059188 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.059234 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":37}
I20260808 00:35:11.059342 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.055431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:42683 (local address 127.31.210.195:38671)
0808 00:35:11.055638 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.055648 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.055704 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.055840 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.055841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.055842 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.055952 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.055954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.057001 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.057002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.058213 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.058215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.059026 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.059027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.059117 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.059122 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.059188 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.059234 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":66,"thread_start_us":141,"threads_started":1}
I20260808 00:35:11.059741 20316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1465cda2db42dfbac3ef1ee16a1623" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f432c21575f84cfc9c5cac352c8f482b" is_pre_election: true
I20260808 00:35:11.059943 20316 raft_consensus.cc:2454] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4dce29fc0e4ee38e23d049f48eb14a in term 0.
I20260808 00:35:11.059974 20112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1465cda2db42dfbac3ef1ee16a1623" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da51014a186241b4b98612265fc560a9" is_pre_election: true
I20260808 00:35:11.060166 20112 raft_consensus.cc:2454] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4dce29fc0e4ee38e23d049f48eb14a in term 0.
I20260808 00:35:11.060262 20421 leader_election.cc:302] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b; no voters:
I20260808 00:35:11.060421 20609 raft_consensus.cc:2781] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:35:11.060529 20609 raft_consensus.cc:482] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:35:11.060571 20609 raft_consensus.cc:3037] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:11.061277 20609 raft_consensus.cc:504] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.061432 20609 leader_election.cc:288] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 election: Requested vote from peers f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145), da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:11.061589 20112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1465cda2db42dfbac3ef1ee16a1623" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da51014a186241b4b98612265fc560a9"
I20260808 00:35:11.061630 20316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1465cda2db42dfbac3ef1ee16a1623" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f432c21575f84cfc9c5cac352c8f482b"
I20260808 00:35:11.061683 20112 raft_consensus.cc:3037] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:11.061712 20316 raft_consensus.cc:3037] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:11.062427 20316 raft_consensus.cc:2454] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4dce29fc0e4ee38e23d049f48eb14a in term 1.
I20260808 00:35:11.062500 20112 raft_consensus.cc:2454] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4dce29fc0e4ee38e23d049f48eb14a in term 1.
I20260808 00:35:11.062618 20421 leader_election.cc:302] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b; no voters:
I20260808 00:35:11.062711 20609 raft_consensus.cc:2781] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:11.062858 20609 raft_consensus.cc:686] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 LEADER]: Becoming Leader. State: Replica: de4dce29fc0e4ee38e23d049f48eb14a, State: Running, Role: LEADER
I20260808 00:35:11.062950 20609 consensus_queue.cc:237] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.063800 19921 catalog_manager.cc:5495] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a reported cstate change: term changed from 0 to 1, leader changed from <none> to de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195). New cstate: current_term: 1 leader_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } health_report { overall_health: HEALTHY } } }
W20260808 00:35:11.070083 20384 tablet.cc:2369] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:35:11.080855 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.078661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52620 (local address 127.31.210.195:34309)
0808 00:35:11.078730 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.078732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.078745 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:35:11.078778 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.078779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.078795 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.078859 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:35:11.078972 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.079429 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.080151 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.080333 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.080384 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.080553 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:35:11.080658 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:35:11.080725 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.080760 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.080779 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:35:11.082754 20112 raft_consensus.cc:1261] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Refusing update from remote peer de4dce29fc0e4ee38e23d049f48eb14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:11.082778 20316 raft_consensus.cc:1261] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Refusing update from remote peer de4dce29fc0e4ee38e23d049f48eb14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:11.083137 20609 consensus_queue.cc:1035] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:11.083282 20615 consensus_queue.cc:1035] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:11.085757 20622 mvcc.cc:205] Tried to move back new op lower bound from 7316067578191998976 to 7316067578114469888. Current Snapshot: MvccSnapshot[applied={T|T < 7316067578191998976}]
I20260808 00:35:11.085837 20625 mvcc.cc:205] Tried to move back new op lower bound from 7316067578191998976 to 7316067578114469888. Current Snapshot: MvccSnapshot[applied={T|T < 7316067578191998976}]
I20260808 00:35:11.086622 20620 mvcc.cc:205] Tried to move back new op lower bound from 7316067578191998976 to 7316067578114469888. Current Snapshot: MvccSnapshot[applied={T|T < 7316067578191998976 or (T in {7316067578191998976})}]
I20260808 00:35:11.087756 19921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:11.087855 19921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:11.088665 19921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Sending DeleteTablet for 3 replicas of tablet cc1465cda2db42dfbac3ef1ee16a1623
I20260808 00:35:11.089026 20292 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1465cda2db42dfbac3ef1ee16a1623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:11 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:50678
I20260808 00:35:11.089257 20636 tablet_replica.cc:330] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: stopping tablet replica
I20260808 00:35:11.089257 20479 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1465cda2db42dfbac3ef1ee16a1623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:11 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52604
I20260808 00:35:11.089365 20636 raft_consensus.cc:2229] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:11.089479 20636 raft_consensus.cc:2258] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:11.089460 20099 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1465cda2db42dfbac3ef1ee16a1623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:11 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:55524
I20260808 00:35:11.089766 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 19999
I20260808 00:35:11.089674 20640 tablet_replica.cc:330] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: stopping tablet replica
I20260808 00:35:11.090039 20636 ts_tablet_manager.cc:1905] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:11.095319 20636 ts_tablet_manager.cc:1918] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:11.095497 20636 log.cc:1196] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/cc1465cda2db42dfbac3ef1ee16a1623
I20260808 00:35:11.095885 20636 ts_tablet_manager.cc:1939] T cc1465cda2db42dfbac3ef1ee16a1623 P f432c21575f84cfc9c5cac352c8f482b: Deleting consensus metadata
I20260808 00:35:11.096411 19899 catalog_manager.cc:4841] TS f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145): tablet cc1465cda2db42dfbac3ef1ee16a1623 (table test-table [id=02e30044cf2f4de5baf3968c7dae6506]) successfully deleted
I20260808 00:35:11.099471 20638 tablet_replica.cc:330] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: stopping tablet replica
I20260808 00:35:11.099611 20638 raft_consensus.cc:2229] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:35:11.099769 20638 raft_consensus.cc:2258] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:11.100270 20638 ts_tablet_manager.cc:1905] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:11.102309 20638 ts_tablet_manager.cc:1918] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:11.102396 20638 log.cc:1196] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/cc1465cda2db42dfbac3ef1ee16a1623
I20260808 00:35:11.102685 20638 ts_tablet_manager.cc:1939] T cc1465cda2db42dfbac3ef1ee16a1623 P de4dce29fc0e4ee38e23d049f48eb14a: Deleting consensus metadata
I20260808 00:35:11.103148 19895 catalog_manager.cc:4841] TS de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309): tablet cc1465cda2db42dfbac3ef1ee16a1623 (table test-table [id=02e30044cf2f4de5baf3968c7dae6506]) successfully deleted
W20260808 00:35:11.105643 19895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:42683 (error 108)
I20260808 00:35:11.106204 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.106060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:42683 (local address 127.0.0.1:55528)
0808 00:35:11.106126 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.106167 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 00:35:11.106268 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 20202
W20260808 00:35:11.106326 19895 catalog_manager.cc:4584] TS da51014a186241b4b98612265fc560a9 (127.31.210.193:42683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc1465cda2db42dfbac3ef1ee16a1623: Network error: Client connection negotiation failed: client connection to 127.31.210.193:42683: connect: Connection refused (error 111)
I20260808 00:35:11.121699 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 20391
I20260808 00:35:11.131529 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 19867
I20260808 00:35:11.140574 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44411
--webserver_interface=127.31.210.254
--webserver_port=45825
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:44411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:11.224902 20645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:11.225131 20645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:11.225154 20645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:11.226312 20645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:11.226358 20645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:11.226372 20645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:11.226385 20645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:11.226397 20645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:11.227886 20645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:44411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:11.228174 20645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:11.228413 20645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:11.238369 20652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.238364 20656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.238364 20651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:11.238397 20645 server_base.cc:1034] running on GCE node
I20260808 00:35:11.239070 20645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:11.239394 20645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:11.240549 20645 hybrid_clock.cc:648] HybridClock initialized: now 1786149311240527 us; error 37 us; skew 500 ppm
Aug 08 00:35:11 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:11.243330 20645 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:11.243805 20645 webserver.cc:460] Webserver started at http://127.31.210.254:45825/ using document root <none> and password file <none>
I20260808 00:35:11.244050 20645 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:11.244127 20645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:11.245608 20645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:11.246371 20666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.246577 20645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:11.246680 20645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "19b6d95363a4485fb202574a10fd800a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "a9171d942bc3ae82c2066314a1015906"
server_key_iv: "466a64cdb3adc7010517e108c2c3cab6"
server_key_version: "encryptionkey@0"
I20260808 00:35:11.247084 20645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:11.264940 20645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:11.267414 20645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:11.267594 20645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:11.272024 20645 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:44411
I20260808 00:35:11.272109 20744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:44411 every 8 connection(s)
I20260808 00:35:11.272629 20645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:11.274541 20745 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:35:11.275403 20745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Bootstrap starting.
I20260808 00:35:11.276311 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 20645
I20260808 00:35:11.276657 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:42683
--local_ip_for_outbound_sockets=127.31.210.193
--tserver_master_addrs=127.31.210.254:44411
--webserver_port=45155
--webserver_interface=127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:11.286840 20745 log.cc:824] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:11.296866 20745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:35:11.297271 20745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Bootstrap complete.
I20260808 00:35:11.300945 20745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:11.301276 20745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b6d95363a4485fb202574a10fd800a, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.301554 20745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:11.301692 20745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:35:11.301754 20745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:35:11.301838 20745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:35:11.303678 20745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:11.303819 20745 leader_election.cc:302] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19b6d95363a4485fb202574a10fd800a; no voters:
I20260808 00:35:11.304056 20745 leader_election.cc:288] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:35:11.304137 20752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:35:11.304322 20752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 LEADER]: Becoming Leader. State: Replica: 19b6d95363a4485fb202574a10fd800a, State: Running, Role: LEADER
I20260808 00:35:11.304426 20752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:11.304452 20745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:11.304706 20753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "19b6d95363a4485fb202574a10fd800a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } } }
I20260808 00:35:11.304831 20753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:11.304704 20752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19b6d95363a4485fb202574a10fd800a. Latest consensus state: current_term: 2 leader_uuid: "19b6d95363a4485fb202574a10fd800a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } } }
I20260808 00:35:11.305086 20752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:11.305357 20759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:11.306241 20759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:11.306353 20759 catalog_manager.cc:1259] Loaded cluster ID: c6588ffbb5824493a307f6bfb88728d1
I20260808 00:35:11.306421 20759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:35:11.306931 20759 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 00:35:11.311585 20759 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 00:35:11.314178 20759 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:35:11.314415 20759 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Loaded TSK: 0
I20260808 00:35:11.314632 20759 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:35:11.377012 20750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:11.377254 20750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:11.377293 20750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:11.378511 20750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:11.378582 20750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:11.378630 20750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:35:11.380117 20750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:42683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=45155
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:11.380383 20750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:11.380656 20750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:11.386029 20792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.386176 20795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.386039 20790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:11.386575 20750 server_base.cc:1034] running on GCE node
I20260808 00:35:11.386822 20750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:11.387126 20750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:11.388338 20750 hybrid_clock.cc:648] HybridClock initialized: now 1786149311388310 us; error 45 us; skew 500 ppm
Aug 08 00:35:11 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:11.391628 20750 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:35:11.392198 20750 webserver.cc:460] Webserver started at http://127.31.210.193:45155/ using document root <none> and password file <none>
I20260808 00:35:11.392509 20750 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:11.392621 20750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:11.394487 20750 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:35:11.395442 20805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.395640 20750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:11.395743 20750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "da51014a186241b4b98612265fc560a9"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "21719a0788395cab112facbb0a1623a9"
server_key_iv: "6cdb0483e10c4ac567656602f524dfb6"
server_key_version: "encryptionkey@0"
I20260808 00:35:11.396421 20750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:11.411506 20750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:11.414314 20750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:11.414505 20750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:11.414803 20750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:11.415318 20816 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:35:11.417762 20750 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:35:11.417842 20750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 00:35:11.417891 20750 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:35:11.418629 20750 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:35:11.418694 20750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:35:11.418758 20816 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Bootstrap starting.
I20260808 00:35:11.426852 20750 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:42683
I20260808 00:35:11.427266 20750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:35:11.428750 20980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:42683 every 8 connection(s)
I20260808 00:35:11.430339 20816 log.cc:824] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:11.433286 20816 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:35:11.433627 20816 tablet_bootstrap.cc:492] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Bootstrap complete.
I20260808 00:35:11.434966 20816 ts_tablet_manager.cc:1397] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
Aug 08 00:35:11 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:11.436235 20816 raft_consensus.cc:348] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.436580 20816 raft_consensus.cc:729] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da51014a186241b4b98612265fc560a9, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.436698 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 20750
I20260808 00:35:11.436757 20816 consensus_queue.cc:260] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } } peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } }
I20260808 00:35:11.437029 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:46145
--local_ip_for_outbound_sockets=127.31.210.194
--tserver_master_addrs=127.31.210.254:44411
--webserver_port=38839
--webserver_interface=127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:11.437069 20816 ts_tablet_manager.cc:1428] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260808 00:35:11.441983 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.428196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:37519)
0808 00:35:11.429063 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.429100 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.429578 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.430138 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.430143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.430533 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:11.430865 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.430874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.432172 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.432173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.433187 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.433190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.434110 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.434112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.434249 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.434961 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:11.434971 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:11.437071 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:11.440051 (+ 2980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:11.440058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:11.440061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:11.440368 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:11.440716 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:11.440717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:11.440717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:11.440841 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:11.441333 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:11.441340 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:11.441483 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.441570 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.441755 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":118,"threads_started":1}
I20260808 00:35:11.442584 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.428273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:37519 (local address 127.31.210.254:44411)
0808 00:35:11.429752 (+ 1479us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.429760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.429789 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:35:11.429901 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.429907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.429949 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.430103 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:11.431003 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.432025 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.433308 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.433882 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.434000 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.437453 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:11.437458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:11.437459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:11.437472 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:11.439773 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.440471 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.440474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.440474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.440620 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.440938 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.440939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.440939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.441238 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:11.442129 (+ 891us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.442225 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.442371 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1429,"thread_start_us":89,"threads_started":1}
I20260808 00:35:11.443969 20981 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44411
I20260808 00:35:11.444104 20981 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:11.444298 20981 heartbeater.cc:511] Master 127.31.210.254:44411 requested a full tablet report, sending...
I20260808 00:35:11.445179 20692 ts_manager.cc:194] Registered new tserver with Master: da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:11.446426 20692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:37519
I20260808 00:35:11.447572 20981 heartbeater.cc:503] Master 127.31.210.254:44411 was elected leader, sending a full tablet report...
I20260808 00:35:11.453799 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.445861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:42683 (local address 127.0.0.1:55538)
0808 00:35:11.448624 (+ 2763us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.448663 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.448736 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.449067 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.449071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.449074 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.449156 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.449159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.450695 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.450696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.452272 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.452276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.453493 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.453494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.453589 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.453605 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.453654 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.453709 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2726,"thread_start_us":98,"threads_started":1}
I20260808 00:35:11.454336 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.446153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55538 (local address 127.31.210.193:42683)
0808 00:35:11.448884 (+ 2731us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.448888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.448898 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:35:11.448933 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.448935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.448951 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.449033 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.449297 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.450569 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.452413 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.453354 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.454117 (+ 763us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.454192 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.454223 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.454265 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2689,"thread_start_us":88,"threads_started":1}
I20260808 00:35:11.454873 20884 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1465cda2db42dfbac3ef1ee16a1623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:11 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:55538
I20260808 00:35:11.455513 20991 tablet_replica.cc:330] stopping tablet replica
I20260808 00:35:11.455638 20991 raft_consensus.cc:2229] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:11.455737 20991 raft_consensus.cc:2258] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:11.456097 20991 ts_tablet_manager.cc:1905] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:11.458439 20991 ts_tablet_manager.cc:1918] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:11.458585 20991 log.cc:1196] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/cc1465cda2db42dfbac3ef1ee16a1623
I20260808 00:35:11.458877 20991 ts_tablet_manager.cc:1939] T cc1465cda2db42dfbac3ef1ee16a1623 P da51014a186241b4b98612265fc560a9: Deleting consensus metadata
I20260808 00:35:11.459476 20668 catalog_manager.cc:4841] TS da51014a186241b4b98612265fc560a9 (127.31.210.193:42683): tablet cc1465cda2db42dfbac3ef1ee16a1623 (table test-table [id=02e30044cf2f4de5baf3968c7dae6506]) successfully deleted
W20260808 00:35:11.530341 20988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:11.530599 20988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:11.530643 20988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:11.531853 20988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:11.531929 20988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:11.531977 20988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:35:11.533541 20988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:46145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=38839
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:11.533870 20988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:11.534123 20988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:11.539268 20998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.539263 21000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.539263 20997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:11.539597 20988 server_base.cc:1034] running on GCE node
I20260808 00:35:11.539812 20988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:11.540033 20988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:11.541249 20988 hybrid_clock.cc:648] HybridClock initialized: now 1786149311541234 us; error 27 us; skew 500 ppm
Aug 08 00:35:11 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:11.544106 20988 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:35:11.544652 20988 webserver.cc:460] Webserver started at http://127.31.210.194:38839/ using document root <none> and password file <none>
I20260808 00:35:11.544893 20988 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:11.544961 20988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:11.546442 20988 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:35:11.547143 21014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.547474 20988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:35:11.547586 20988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "f432c21575f84cfc9c5cac352c8f482b"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "0ca6aa7ef212ecaa015c1082f3c696f3"
server_key_iv: "db8d6b165a96667973ab035924450a7e"
server_key_version: "encryptionkey@0"
I20260808 00:35:11.547952 20988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:11.575116 20988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:11.577483 20988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:11.577661 20988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:11.577922 20988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:11.578255 20988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:11.578310 20988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.578353 20988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:11.578398 20988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.584611 20988 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:46145
I20260808 00:35:11.584723 21182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:46145 every 8 connection(s)
I20260808 00:35:11.585069 20988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:35:11.586380 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 20988
I20260808 00:35:11.586678 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:34309
--local_ip_for_outbound_sockets=127.31.210.195
--tserver_master_addrs=127.31.210.254:44411
--webserver_port=36683
--webserver_interface=127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:35:11 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:11.597185 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.585996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:47939)
0808 00:35:11.586155 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.586217 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.587031 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.587304 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.587309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.588182 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:11.588508 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.588514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.589713 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.589715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.590529 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.590531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.591058 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.591059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.591179 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.591802 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:11.591816 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:11.593599 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:11.595673 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:11.595681 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:11.595683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:11.595911 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:11.596163 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:11.596164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:11.596164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:11.596269 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:11.596643 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:11.596650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:11.596775 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.596865 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.596995 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260808 00:35:11.597172 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.586003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:47939 (local address 127.31.210.254:44411)
0808 00:35:11.586069 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.586071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.586323 (+ 252us) server_negotiation.cc:407] Connection header received
0808 00:35:11.587161 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.587163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.587177 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.587221 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:11.588688 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.589591 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.590622 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.590961 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.591054 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.593711 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:11.593713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:11.593714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:11.593725 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:11.595548 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.595994 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.595996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.595996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.596077 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.596339 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.596340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.596341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.596564 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:11.596657 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.596920 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.597032 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:35:11.597781 21186 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44411
I20260808 00:35:11.597913 21186 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:11.598155 21186 heartbeater.cc:511] Master 127.31.210.254:44411 requested a full tablet report, sending...
I20260808 00:35:11.598601 20692 ts_manager.cc:194] Registered new tserver with Master: f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
I20260808 00:35:11.598994 20692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:47939
W20260808 00:35:11.676837 21190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:11.677068 21190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:11.677093 21190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:11.678272 21190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:11.678321 21190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:11.678385 21190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:35:11.679838 21190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:34309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=36683
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:11.680233 21190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:11.680509 21190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:11.685819 21201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.685829 21197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:11.685829 21198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:11.686080 21190 server_base.cc:1034] running on GCE node
I20260808 00:35:11.686396 21190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:11.686653 21190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:11.687846 21190 hybrid_clock.cc:648] HybridClock initialized: now 1786149311687840 us; error 41 us; skew 500 ppm
Aug 08 00:35:11 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:11.690696 21190 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:35:11.691197 21190 webserver.cc:460] Webserver started at http://127.31.210.195:36683/ using document root <none> and password file <none>
I20260808 00:35:11.691453 21190 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:11.691543 21190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:11.693166 21190 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:35:11.693907 21217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.694140 21190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:11.694262 21190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "de4dce29fc0e4ee38e23d049f48eb14a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "db89474d913664f5a1cd9b587969da83"
server_key_iv: "98618e7367a360407f73bf9d69604727"
server_key_version: "encryptionkey@0"
I20260808 00:35:11.694633 21190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:11.709656 21190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:11.712342 21190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:11.712612 21190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:11.712914 21190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:11.713272 21190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:11.713330 21190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.713371 21190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:11.713418 21190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:11.720252 21190 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:34309
I20260808 00:35:11.720309 21374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:34309 every 8 connection(s)
I20260808 00:35:11.720707 21190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:35:11.724053 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 21190
Aug 08 00:35:11 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149311, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:11.732429 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.721434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:54101 (local address 127.31.210.254:44411)
0808 00:35:11.721558 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.721561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.721755 (+ 194us) server_negotiation.cc:407] Connection header received
0808 00:35:11.722319 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.722320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.722335 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.722418 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:11.723496 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.724177 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.725506 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.725728 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.725822 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.728584 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:11.728585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:11.728586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:11.728594 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:11.730414 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.731045 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.731046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.731047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.731138 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.731494 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.731495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.731495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.731642 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:11.731728 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.732199 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.732305 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 00:35:11.732738 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.724580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47356 (local address 127.31.210.254:44411)
0808 00:35:11.724702 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.724703 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.724817 (+ 114us) server_negotiation.cc:407] Connection header received
0808 00:35:11.724872 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.724873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.724887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.724974 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:11.725653 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.726243 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.727395 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.727596 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.727675 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.729258 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:11.729259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:11.729260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:11.729273 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:11.731319 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.731814 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.731815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.731816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.731885 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:11.732063 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:11.732064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:11.732065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:11.732216 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:11.732415 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.732461 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.732590 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260808 00:35:11.732851 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.721422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:54101)
0808 00:35:11.721629 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.721667 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.722207 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.722501 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.722505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.723161 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:11.723376 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.723382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.724271 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.724272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.725398 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.725401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.725837 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.725838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.725979 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.726590 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:11.726601 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:11.728397 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:11.730538 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:11.730547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:11.730549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:11.730921 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:11.731230 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:11.731232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:11.731232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:11.731403 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:11.731718 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:11.731723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:11.732017 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.732168 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.732387 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260808 00:35:11.733367 21375 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44411
I20260808 00:35:11.733516 21375 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:11.733740 21375 heartbeater.cc:511] Master 127.31.210.254:44411 requested a full tablet report, sending...
I20260808 00:35:11.734184 20692 ts_manager.cc:194] Registered new tserver with Master: de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309)
I20260808 00:35:11.734509 20692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:54101
I20260808 00:35:11.745036 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:35:11.747871 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.745394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.31.210.254:44411)
0808 00:35:11.745449 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.745451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.745641 (+ 190us) server_negotiation.cc:407] Connection header received
0808 00:35:11.745701 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.745701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.745736 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.745798 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:35:11.745902 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.746371 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.747094 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.747290 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.747346 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.747491 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:35:11.747612 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:35:11.747677 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.747713 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.747754 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:35:11.748190 20692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:35:11.750015 20692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:35:11.752504 20884 tablet_service.cc:1467] Processing CreateTablet for tablet fca366819b1945729262f6ccacaf6081 (DEFAULT_TABLE table=test-table [id=d3d20add5cfa4e3a8b53498473c0537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:11.752758 20884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca366819b1945729262f6ccacaf6081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:11.754443 20816 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Bootstrap starting.
I20260808 00:35:11.754778 20816 tablet_bootstrap.cc:654] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:11.755420 20816 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: No bootstrap required, opened a new log
I20260808 00:35:11.755478 20816 ts_tablet_manager.cc:1397] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:11.755811 20816 raft_consensus.cc:348] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:11.755899 20816 raft_consensus.cc:374] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:11.755970 20816 raft_consensus.cc:729] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da51014a186241b4b98612265fc560a9, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.756037 20816 consensus_queue.cc:260] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:11.756145 20816 ts_tablet_manager.cc:1428] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:11.756383 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.752211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.31.210.195:34309)
0808 00:35:11.752431 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.752444 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.752666 (+ 222us) server_negotiation.cc:407] Connection header received
0808 00:35:11.752801 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.752804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.752822 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.752900 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.753086 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.753779 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.755098 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.755887 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.755966 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.756032 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.756248 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.756295 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":138,"threads_started":1}
I20260808 00:35:11.756471 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.752083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34309 (local address 127.0.0.1:52622)
0808 00:35:11.752576 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.752587 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.752697 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.752979 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.752979 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.752981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.753017 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.753018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.753889 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.753891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.754985 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.754987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.756063 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.756064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.756135 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.756139 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.756327 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.756372 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":20,"thread_start_us":56,"threads_started":1}
I20260808 00:35:11.757221 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.752084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:46145 (local address 127.0.0.1:50684)
0808 00:35:11.752207 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:11.752219 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:11.752271 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:11.752929 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:11.752930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:11.752931 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:11.752975 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.752976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.754165 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.754166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.755175 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:11.755177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.757009 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.757010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:11.757077 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.757081 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:35:11.757115 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 00:35:11.757160 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 00:35:11.757373 21284 tablet_service.cc:1467] Processing CreateTablet for tablet fca366819b1945729262f6ccacaf6081 (DEFAULT_TABLE table=test-table [id=d3d20add5cfa4e3a8b53498473c0537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:11.757666 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:11.752530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.31.210.194:46145)
0808 00:35:11.752765 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:11.752770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:11.752782 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:35:11.752824 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:11.752826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:11.752844 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:11.752933 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:11.753044 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.754061 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.756125 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:11.756912 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:11.757424 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:11.757491 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:11.757513 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 00:35:11.757568 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":108,"threads_started":1}
I20260808 00:35:11.757776 21284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca366819b1945729262f6ccacaf6081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:11.758351 21091 tablet_service.cc:1467] Processing CreateTablet for tablet fca366819b1945729262f6ccacaf6081 (DEFAULT_TABLE table=test-table [id=d3d20add5cfa4e3a8b53498473c0537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:11.758669 21091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca366819b1945729262f6ccacaf6081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:11.760982 21391 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Bootstrap starting.
I20260808 00:35:11.761556 21392 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Bootstrap starting.
I20260808 00:35:11.761726 21391 tablet_bootstrap.cc:654] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:11.762173 21392 tablet_bootstrap.cc:654] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:11.762235 21391 log.cc:824] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:11.762683 21392 log.cc:824] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:11.763002 21391 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: No bootstrap required, opened a new log
I20260808 00:35:11.763088 21391 ts_tablet_manager.cc:1397] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:11.763513 21392 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: No bootstrap required, opened a new log
I20260808 00:35:11.763594 21392 ts_tablet_manager.cc:1397] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:35:11.764711 21391 raft_consensus.cc:348] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:11.764868 21391 raft_consensus.cc:374] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:11.764911 21391 raft_consensus.cc:729] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4dce29fc0e4ee38e23d049f48eb14a, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.765040 21391 consensus_queue.cc:260] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:11.764999 21392 raft_consensus.cc:348] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:11.765117 21392 raft_consensus.cc:374] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:11.765139 21392 raft_consensus.cc:729] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f432c21575f84cfc9c5cac352c8f482b, State: Initialized, Role: FOLLOWER
I20260808 00:35:11.765244 21392 consensus_queue.cc:260] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:11.765352 21391 ts_tablet_manager.cc:1428] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:35:11.765419 21375 heartbeater.cc:503] Master 127.31.210.254:44411 was elected leader, sending a full tablet report...
I20260808 00:35:11.765527 21392 ts_tablet_manager.cc:1428] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:35:11.765690 21186 heartbeater.cc:503] Master 127.31.210.254:44411 was elected leader, sending a full tablet report...
W20260808 00:35:11.835687 21188 tablet.cc:2369] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:35:11.929383 20982 tablet.cc:2369] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:35:11.971369 21376 tablet.cc:2369] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:35:12.056208 21395 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:12.056403 21395 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.056944 21395 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145), da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:12.057518 21396 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:12.057723 21396 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.058249 21396 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:12.059753 20987 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:12.059864 20987 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.060195 20987 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
I20260808 00:35:12.061199 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.057255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:42683 (local address 127.31.210.195:48059)
0808 00:35:12.057326 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.057341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:12.057388 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:12.057703 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:12.057704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:12.057706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:12.057783 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.057785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.058744 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.058746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.060029 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.060032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.060954 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.060955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.061048 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.061052 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:35:12.061081 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 00:35:12.061118 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 00:35:12.061499 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.057352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:48059 (local address 127.31.210.193:42683)
0808 00:35:12.057497 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:12.057499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:12.057508 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:35:12.057547 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:12.057548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:12.057558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:12.057694 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:12.057866 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.058644 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.060120 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.060861 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.061369 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.061379 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:12.061397 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 00:35:12.061434 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260808 00:35:12.061691 20913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da51014a186241b4b98612265fc560a9" is_pre_election: true
I20260808 00:35:12.061793 20913 raft_consensus.cc:2454] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4dce29fc0e4ee38e23d049f48eb14a in term 0.
I20260808 00:35:12.062012 21220 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da51014a186241b4b98612265fc560a9, de4dce29fc0e4ee38e23d049f48eb14a; no voters:
I20260808 00:35:12.062167 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.058513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:42683 (local address 127.31.210.194:54725)
0808 00:35:12.058743 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.058755 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:12.058809 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:12.059022 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:12.059022 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:12.059024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:12.059087 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.059088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.059970 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.059971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.061161 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.061164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.061906 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.061907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.061979 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.061983 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:35:12.062034 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:35:12.062078 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":62,"threads_started":1}
I20260808 00:35:12.062155 21395 raft_consensus.cc:2781] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:35:12.062202 21395 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:35:12.062230 21395 raft_consensus.cc:3037] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:12.062443 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.058631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:54725 (local address 127.31.210.193:42683)
0808 00:35:12.058879 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:12.058881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:12.058889 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:35:12.058909 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:12.058910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:12.058920 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:12.058980 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:12.059160 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.059876 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.061253 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.061788 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.061863 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.061871 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:12.062310 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 00:35:12.062361 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":52,"threads_started":1}
I20260808 00:35:12.062525 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.057367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:37653 (local address 127.31.210.194:46145)
0808 00:35:12.057444 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:12.057447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:12.057605 (+ 158us) server_negotiation.cc:407] Connection header received
0808 00:35:12.057748 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:12.057750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:12.057761 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:12.057797 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:12.058089 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.058774 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.061509 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.062110 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.062159 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.062167 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:12.062417 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 00:35:12.062457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:35:12.062742 20913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "f432c21575f84cfc9c5cac352c8f482b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da51014a186241b4b98612265fc560a9" is_pre_election: true
I20260808 00:35:12.062834 20913 raft_consensus.cc:2454] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f432c21575f84cfc9c5cac352c8f482b in term 0.
I20260808 00:35:12.063068 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.057267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:46145 (local address 127.31.210.195:37653)
0808 00:35:12.057500 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.057514 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:12.057645 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:12.057918 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:12.057919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:12.057921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:12.058000 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.058002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.058894 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.058895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.061411 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.061414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.062240 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.062242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.062334 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.062338 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:35:12.062901 (+ 563us) client_negotiation.cc:240] Negotiation successful
0808 00:35:12.062961 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260808 00:35:12.063056 21017 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da51014a186241b4b98612265fc560a9, f432c21575f84cfc9c5cac352c8f482b; no voters:
I20260808 00:35:12.063195 21396 raft_consensus.cc:2781] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:35:12.063277 21396 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:35:12.063315 21396 raft_consensus.cc:3037] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:12.063251 21395 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.063413 21395 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 election: Requested vote from peers f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145), da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:12.063421 21117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f432c21575f84cfc9c5cac352c8f482b" is_pre_election: true
I20260808 00:35:12.063586 20913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da51014a186241b4b98612265fc560a9"
I20260808 00:35:12.063611 21116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f432c21575f84cfc9c5cac352c8f482b"
I20260808 00:35:12.063644 20913 raft_consensus.cc:3037] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:12.064118 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.058699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:60767 (local address 127.31.210.195:34309)
0808 00:35:12.058743 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:12.058744 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:12.058753 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:35:12.058779 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:12.058780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:12.058789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:12.058849 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:12.059041 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.059803 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.061518 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.062090 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.062165 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.062172 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:12.064016 (+ 1844us) server_negotiation.cc:299] Negotiation successful
0808 00:35:12.064059 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":158,"server-negotiator.queue_time_us":24}
I20260808 00:35:12.064291 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.058378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34309 (local address 127.31.210.194:60767)
0808 00:35:12.058419 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.058430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:12.058473 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:12.058880 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:12.058881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:12.058883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:12.058950 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.058951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.059933 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.059934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.061205 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.061208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.062819 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.062820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.062905 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.062910 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:12.062970 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:35:12.063009 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":19}
I20260808 00:35:12.064308 21396 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.064471 20913 raft_consensus.cc:2454] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4dce29fc0e4ee38e23d049f48eb14a in term 1.
I20260808 00:35:12.064468 21396 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [CANDIDATE]: Term 1 election: Requested vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), da51014a186241b4b98612265fc560a9 (127.31.210.193:42683)
I20260808 00:35:12.064584 21117 raft_consensus.cc:2379] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de4dce29fc0e4ee38e23d049f48eb14a in current term 1: Already voted for candidate f432c21575f84cfc9c5cac352c8f482b in this term.
I20260808 00:35:12.064636 21220 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: da51014a186241b4b98612265fc560a9, de4dce29fc0e4ee38e23d049f48eb14a; no voters: f432c21575f84cfc9c5cac352c8f482b
I20260808 00:35:12.064642 21315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "f432c21575f84cfc9c5cac352c8f482b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" is_pre_election: true
I20260808 00:35:12.064807 21314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "f432c21575f84cfc9c5cac352c8f482b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4dce29fc0e4ee38e23d049f48eb14a"
I20260808 00:35:12.064767 21395 raft_consensus.cc:2781] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:12.065178 21395 raft_consensus.cc:686] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 LEADER]: Becoming Leader. State: Replica: de4dce29fc0e4ee38e23d049f48eb14a, State: Running, Role: LEADER
I20260808 00:35:12.065299 20913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "f432c21575f84cfc9c5cac352c8f482b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da51014a186241b4b98612265fc560a9"
I20260808 00:35:12.065303 21395 consensus_queue.cc:237] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.065409 20913 raft_consensus.cc:2379] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f432c21575f84cfc9c5cac352c8f482b in current term 1: Already voted for candidate de4dce29fc0e4ee38e23d049f48eb14a in this term.
I20260808 00:35:12.065611 21017 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f432c21575f84cfc9c5cac352c8f482b; no voters: da51014a186241b4b98612265fc560a9, de4dce29fc0e4ee38e23d049f48eb14a
I20260808 00:35:12.065768 21396 raft_consensus.cc:2726] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 00:35:12.066293 20691 catalog_manager.cc:5495] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a reported cstate change: term changed from 0 to 1, leader changed from <none> to de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195). New cstate: current_term: 1 leader_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } health_report { overall_health: UNKNOWN } } }
I20260808 00:35:12.068670 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.060757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:53121 (local address 127.31.210.194:46145)
0808 00:35:12.062606 (+ 1849us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:12.062607 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:12.065426 (+ 2819us) server_negotiation.cc:407] Connection header received
0808 00:35:12.065450 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:12.065452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:12.065469 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:12.065516 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:12.065822 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.066531 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.067893 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.068244 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.068298 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.068303 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:12.068551 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 00:35:12.068594 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1835,"thread_start_us":77,"threads_started":1}
I20260808 00:35:12.068713 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.060597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:46145 (local address 127.31.210.193:53121)
0808 00:35:12.065310 (+ 4713us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.065320 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:12.065384 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:12.065572 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:12.065573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:12.065575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:12.065739 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.065742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.066625 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.066627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.067799 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.067802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.068350 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.068351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.068433 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.068438 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:12.068538 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 00:35:12.068595 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4700,"thread_start_us":44,"threads_started":1}
I20260808 00:35:12.068758 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.060726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:51455 (local address 127.31.210.195:34309)
0808 00:35:12.064209 (+ 3483us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:12.064211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:12.065141 (+ 930us) server_negotiation.cc:407] Connection header received
0808 00:35:12.065163 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:12.065164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:12.065175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:12.065222 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:12.066023 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.066668 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.067940 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.068329 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.068410 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.068419 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:12.068650 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 00:35:12.068688 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3468,"thread_start_us":87,"threads_started":1}
I20260808 00:35:12.068976 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.060382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34309 (local address 127.31.210.193:51455)
0808 00:35:12.064999 (+ 4617us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.065013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:12.065093 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:12.065859 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:12.065860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:12.065863 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:12.065921 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.065922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.066758 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.066759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.067808 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:12.067811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.068426 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.068427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:12.068554 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.068559 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:12.068828 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 00:35:12.068886 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4586,"mutex_wait_us":4,"thread_start_us":66,"threads_started":1}
I20260808 00:35:12.069168 21117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "da51014a186241b4b98612265fc560a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f432c21575f84cfc9c5cac352c8f482b" is_pre_election: true
I20260808 00:35:12.069273 21117 raft_consensus.cc:2379] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da51014a186241b4b98612265fc560a9 in current term 1: Already voted for candidate f432c21575f84cfc9c5cac352c8f482b in this term.
I20260808 00:35:12.069271 21315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca366819b1945729262f6ccacaf6081" candidate_uuid: "da51014a186241b4b98612265fc560a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" is_pre_election: true
I20260808 00:35:12.069520 20808 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da51014a186241b4b98612265fc560a9; no voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b
I20260808 00:35:12.069635 20987 raft_consensus.cc:2726] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 00:35:12.112886 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.110492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.31.210.195:34309)
0808 00:35:12.110549 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:12.110550 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:12.110557 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:35:12.110594 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:12.110596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:12.110616 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:12.110681 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:35:12.110797 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.111313 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.112158 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:12.112347 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:12.112405 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:12.112572 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:35:12.112680 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:35:12.112743 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:12.112780 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 00:35:12.112797 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:35:12.114918 20913 raft_consensus.cc:1261] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Refusing update from remote peer de4dce29fc0e4ee38e23d049f48eb14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:12.114918 21117 raft_consensus.cc:1261] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Refusing update from remote peer de4dce29fc0e4ee38e23d049f48eb14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:12.115202 21414 consensus_queue.cc:1035] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:12.115389 21395 consensus_queue.cc:1035] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:12.117970 21417 mvcc.cc:205] Tried to move back new op lower bound from 7316067582419574784 to 7316067582220165120. Current Snapshot: MvccSnapshot[applied={T|T < 7316067582419574784}]
I20260808 00:35:12.118207 21420 mvcc.cc:205] Tried to move back new op lower bound from 7316067582419574784 to 7316067582220165120. Current Snapshot: MvccSnapshot[applied={T|T < 7316067582419574784}]
I20260808 00:35:12.119879 20691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:12.119982 20691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:12.120731 20691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Sending DeleteTablet for 3 replicas of tablet fca366819b1945729262f6ccacaf6081
I20260808 00:35:12.121085 21091 tablet_service.cc:1514] Processing DeleteTablet for tablet fca366819b1945729262f6ccacaf6081 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:12 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:50684
I20260808 00:35:12.121116 21284 tablet_service.cc:1514] Processing DeleteTablet for tablet fca366819b1945729262f6ccacaf6081 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:12 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52622
I20260808 00:35:12.121145 20884 tablet_service.cc:1514] Processing DeleteTablet for tablet fca366819b1945729262f6ccacaf6081 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:12 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:55538
I20260808 00:35:12.121323 21430 tablet_replica.cc:330] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: stopping tablet replica
I20260808 00:35:12.121423 21430 raft_consensus.cc:2229] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:35:12.121421 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 20750
I20260808 00:35:12.121541 21430 raft_consensus.cc:2258] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:12.122028 21430 ts_tablet_manager.cc:1905] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:12.123975 21431 tablet_replica.cc:330] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: stopping tablet replica
I20260808 00:35:12.124073 21431 raft_consensus.cc:2229] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:12.124174 21431 raft_consensus.cc:2258] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:12.125988 21430 ts_tablet_manager.cc:1918] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:12.126101 21430 log.cc:1196] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/fca366819b1945729262f6ccacaf6081
I20260808 00:35:12.126386 21430 ts_tablet_manager.cc:1939] T fca366819b1945729262f6ccacaf6081 P de4dce29fc0e4ee38e23d049f48eb14a: Deleting consensus metadata
I20260808 00:35:12.126891 20668 catalog_manager.cc:4841] TS de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309): tablet fca366819b1945729262f6ccacaf6081 (table test-table [id=d3d20add5cfa4e3a8b53498473c0537c]) successfully deleted
I20260808 00:35:12.127247 21431 ts_tablet_manager.cc:1905] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:12.128844 21431 ts_tablet_manager.cc:1918] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:12.128937 21431 log.cc:1196] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/fca366819b1945729262f6ccacaf6081
I20260808 00:35:12.129215 21431 ts_tablet_manager.cc:1939] T fca366819b1945729262f6ccacaf6081 P f432c21575f84cfc9c5cac352c8f482b: Deleting consensus metadata
I20260808 00:35:12.130210 20671 catalog_manager.cc:4841] TS f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145): tablet fca366819b1945729262f6ccacaf6081 (table test-table [id=d3d20add5cfa4e3a8b53498473c0537c]) successfully deleted
W20260808 00:35:12.138439 20668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:42683 (error 108)
I20260808 00:35:12.139070 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.138862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:42683 (local address 127.0.0.1:55554)
0808 00:35:12.138985 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.139028 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 00:35:12.139211 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 20988
W20260808 00:35:12.139190 20668 catalog_manager.cc:4584] TS da51014a186241b4b98612265fc560a9 (127.31.210.193:42683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fca366819b1945729262f6ccacaf6081: Network error: Client connection negotiation failed: client connection to 127.31.210.193:42683: connect: Connection refused (error 111)
I20260808 00:35:12.153911 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 21190
I20260808 00:35:12.164973 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 20645
I20260808 00:35:12.174604 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44411
--webserver_interface=127.31.210.254
--webserver_port=45825
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:44411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:12.260927 21434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:12.261138 21434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:12.261163 21434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:12.262354 21434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:12.262430 21434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:12.262449 21434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:12.262511 21434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:12.262540 21434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:12.264099 21434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:44411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:12.264376 21434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:12.264655 21434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:12.269929 21443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:12.269989 21441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:12.269894 21450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:12.270049 21434 server_base.cc:1034] running on GCE node
I20260808 00:35:12.270483 21434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:12.270738 21434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:12.271885 21434 hybrid_clock.cc:648] HybridClock initialized: now 1786149312271874 us; error 30 us; skew 500 ppm
Aug 08 00:35:12 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:12.274788 21434 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:12.275332 21434 webserver.cc:460] Webserver started at http://127.31.210.254:45825/ using document root <none> and password file <none>
I20260808 00:35:12.275619 21434 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:12.275672 21434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:12.277277 21434 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:35:12.278095 21459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.278302 21434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:12.278404 21434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "19b6d95363a4485fb202574a10fd800a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "a9171d942bc3ae82c2066314a1015906"
server_key_iv: "466a64cdb3adc7010517e108c2c3cab6"
server_key_version: "encryptionkey@0"
I20260808 00:35:12.278789 21434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:12.298956 21434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:12.301275 21434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:12.301453 21434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:12.305683 21434 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:44411
I20260808 00:35:12.305720 21539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:44411 every 8 connection(s)
I20260808 00:35:12.306217 21434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:12.307915 21540 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:35:12.308710 21540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Bootstrap starting.
I20260808 00:35:12.310609 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 21434
I20260808 00:35:12.310892 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:42683
--local_ip_for_outbound_sockets=127.31.210.193
--tserver_master_addrs=127.31.210.254:44411
--webserver_port=45155
--webserver_interface=127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:12.317668 21540 log.cc:824] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:12.320040 21540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:35:12.320371 21540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: Bootstrap complete.
I20260808 00:35:12.322777 21540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:12.323038 21540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b6d95363a4485fb202574a10fd800a, State: Initialized, Role: FOLLOWER
I20260808 00:35:12.323254 21540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:12.323356 21540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:35:12.323407 21540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:35:12.323457 21540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:35:12.324471 21540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:12.324649 21540 leader_election.cc:302] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19b6d95363a4485fb202574a10fd800a; no voters:
I20260808 00:35:12.324851 21540 leader_election.cc:288] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 00:35:12.324924 21546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:35:12.325084 21546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [term 3 LEADER]: Becoming Leader. State: Replica: 19b6d95363a4485fb202574a10fd800a, State: Running, Role: LEADER
I20260808 00:35:12.325148 21540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:12.325901 21546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } }
I20260808 00:35:12.326556 21550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "19b6d95363a4485fb202574a10fd800a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } } }
I20260808 00:35:12.326615 21549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19b6d95363a4485fb202574a10fd800a. Latest consensus state: current_term: 3 leader_uuid: "19b6d95363a4485fb202574a10fd800a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b6d95363a4485fb202574a10fd800a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44411 } } }
I20260808 00:35:12.326761 21550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:12.326766 21549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:12.327263 21569 catalog_manager.cc:1259] Loaded cluster ID: c6588ffbb5824493a307f6bfb88728d1
I20260808 00:35:12.327313 21569 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: loading cluster ID for follower catalog manager: success
W20260808 00:35:12.327612 21569 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 19b6d95363a4485fb202574a10fd800a: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260808 00:35:12.327677 21569 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260808 00:35:12.328615 21571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:12.328985 21571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:12.329090 21571 catalog_manager.cc:1259] Loaded cluster ID: c6588ffbb5824493a307f6bfb88728d1
I20260808 00:35:12.329124 21571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 00:35:12.329326 21571 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff36eb539ad google::LogMessage::Fail() at ??:0
@ 0x7ff36eb5688a google::LogMessage::SendToLog() at ??:0
@ 0x7ff36eb534de google::LogMessage::Flush() at ??:0
@ 0x7ff36eb54a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff376a740f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ff376a7d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff370a210d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff370a16954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff37187a6db start_thread at ??:0
@ 0x7ff36d65161f clone at ??:0
W20260808 00:35:12.431648 21542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:12.431933 21542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:12.431972 21542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:12.433866 21542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:12.433934 21542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:12.433984 21542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:35:12.436163 21542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:42683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=45155
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:12.436439 21542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:12.436754 21542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:12.442198 21598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:12.442207 21597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:12.442216 21603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:12.442488 21542 server_base.cc:1034] running on GCE node
I20260808 00:35:12.442749 21542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:12.443002 21542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:12.444172 21542 hybrid_clock.cc:648] HybridClock initialized: now 1786149312444145 us; error 42 us; skew 500 ppm
Aug 08 00:35:12 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:12.447858 21542 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:35:12.448349 21542 webserver.cc:460] Webserver started at http://127.31.210.193:45155/ using document root <none> and password file <none>
I20260808 00:35:12.448657 21542 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:12.448726 21542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:12.450135 21542 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:12.450893 21612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.451094 21542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:12.451210 21542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "da51014a186241b4b98612265fc560a9"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "21719a0788395cab112facbb0a1623a9"
server_key_iv: "6cdb0483e10c4ac567656602f524dfb6"
server_key_version: "encryptionkey@0"
I20260808 00:35:12.451630 21542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:12.468689 21542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:12.471210 21542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:12.471380 21542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:12.471625 21542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:12.472093 21623 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:35:12.474336 21542 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:35:12.474398 21542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:35:12.474444 21542 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:35:12.475092 21542 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:35:12.475149 21542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:35:12.475229 21623 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Bootstrap starting.
I20260808 00:35:12.482594 21542 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:42683
I20260808 00:35:12.482722 21778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:42683 every 8 connection(s)
I20260808 00:35:12.483042 21542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:35:12.487907 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 21542
I20260808 00:35:12.488230 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:46145
--local_ip_for_outbound_sockets=127.31.210.194
--tserver_master_addrs=127.31.210.254:44411
--webserver_port=38839
--webserver_interface=127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:12.488315 21623 log.cc:824] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:12.491340 21623 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:35:12.491717 21623 tablet_bootstrap.cc:492] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Bootstrap complete.
I20260808 00:35:12.493273 21623 ts_tablet_manager.cc:1397] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.012s
I20260808 00:35:12.495005 21623 raft_consensus.cc:348] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.495371 21623 raft_consensus.cc:729] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da51014a186241b4b98612265fc560a9, State: Initialized, Role: FOLLOWER
I20260808 00:35:12.495564 21623 consensus_queue.cc:260] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:12.495908 21623 ts_tablet_manager.cc:1428] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260808 00:35:12.563637 21781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:35:12.483803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:55975)
0808 00:35:12.484057 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.484115 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:12.484623 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:12.563414 (+ 78791us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":118,"threads_started":1}
W20260808 00:35:12.563941 21779 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:44411 (0 consecutive failures): Network error: Failed to ping master at 127.31.210.254:44411: Client connection negotiation failed: client connection to 127.31.210.254:44411: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:35:12.564323 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.564194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:38295)
0808 00:35:12.564251 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.564297 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 00:35:12.564772 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.564685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:37657)
0808 00:35:12.564732 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.564755 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 00:35:12.564908 21779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:35:12.576824 21783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:12.577050 21783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:12.577088 21783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:12.578240 21783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:12.578307 21783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:12.578352 21783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:35:12.579833 21783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:46145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=38839
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:12.580096 21783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:12.580331 21783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:12.585490 21791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:12.585510 21790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:12.585695 21783 server_base.cc:1034] running on GCE node
W20260808 00:35:12.585493 21794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:12.585999 21783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:12.586257 21783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:12.587440 21783 hybrid_clock.cc:648] HybridClock initialized: now 1786149312587402 us; error 52 us; skew 500 ppm
Aug 08 00:35:12 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:12.590436 21783 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:35:12.590880 21783 webserver.cc:460] Webserver started at http://127.31.210.194:38839/ using document root <none> and password file <none>
I20260808 00:35:12.591116 21783 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:12.591187 21783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:12.592698 21783 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:12.593417 21808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.593641 21783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:12.593744 21783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "f432c21575f84cfc9c5cac352c8f482b"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "0ca6aa7ef212ecaa015c1082f3c696f3"
server_key_iv: "db8d6b165a96667973ab035924450a7e"
server_key_version: "encryptionkey@0"
I20260808 00:35:12.594134 21783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:12.609053 21783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:12.611039 21783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:12.611210 21783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:12.611459 21783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:12.611781 21783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:12.611833 21783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.611876 21783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:12.611915 21783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.618395 21783 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:46145
I20260808 00:35:12.618467 21962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:46145 every 8 connection(s)
I20260808 00:35:12.618837 21783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:35:12.619777 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 21783
I20260808 00:35:12.620044 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.619521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:54537)
0808 00:35:12.619738 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.619818 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260808 00:35:12.620067 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:34309
--local_ip_for_outbound_sockets=127.31.210.195
--tserver_master_addrs=127.31.210.254:44411
--webserver_port=36683
--webserver_interface=127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:12.620580 21964 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:44411 (0 consecutive failures): Network error: Failed to ping master at 127.31.210.254:44411: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
I20260808 00:35:12.620891 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.620790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:51277)
0808 00:35:12.620842 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.620869 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 00:35:12.621289 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.621210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:34109)
0808 00:35:12.621250 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.621273 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:35:12.621414 21964 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:35:12.707393 21967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:12.707604 21967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:12.707631 21967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:12.708890 21967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:12.708935 21967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:12.708963 21967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:35:12.710456 21967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:34309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=36683
--tserver_master_addrs=127.31.210.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:12.710712 21967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:12.710956 21967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:35:12.716372 21967 server_base.cc:1034] running on GCE node
W20260808 00:35:12.716413 21973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:12.716428 21974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:12.716555 21977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:12.716836 21967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:12.717147 21967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:12.718322 21967 hybrid_clock.cc:648] HybridClock initialized: now 1786149312718303 us; error 40 us; skew 500 ppm
Aug 08 00:35:12 dist-test-slave-44d2 krb5kdc[19845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:12.721240 21967 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:35:12.721714 21967 webserver.cc:460] Webserver started at http://127.31.210.195:36683/ using document root <none> and password file <none>
I20260808 00:35:12.721971 21967 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:12.722039 21967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:12.723356 21967 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:12.723953 21989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.724143 21967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.724272 21967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "de4dce29fc0e4ee38e23d049f48eb14a"
format_stamp: "Formatted at 2026-08-08 00:35:10 on dist-test-slave-44d2"
server_key: "db89474d913664f5a1cd9b587969da83"
server_key_iv: "98618e7367a360407f73bf9d69604727"
server_key_version: "encryptionkey@0"
I20260808 00:35:12.724679 21967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:12.742640 21967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:12.745213 21967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:12.745395 21967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:12.745654 21967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:12.746052 21967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:12.746110 21967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.746160 21967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:12.746187 21967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:12.752653 21967 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:34309
I20260808 00:35:12.752746 22152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:34309 every 8 connection(s)
I20260808 00:35:12.753091 21967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:35:12.754284 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.753891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:58261)
0808 00:35:12.754043 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.754123 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
W20260808 00:35:12.754689 22154 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:44411 (0 consecutive failures): Network error: Failed to ping master at 127.31.210.254:44411: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
I20260808 00:35:12.755036 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.754937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:53441)
0808 00:35:12.754974 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.755011 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 00:35:12.755336 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:12.755263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:49393)
0808 00:35:12.755303 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:12.755323 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 00:35:12.755452 22154 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:35:12.756417 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 21967
I20260808 00:35:13.565516 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:13.565274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:58593)
0808 00:35:13.565402 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:13.565442 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260808 00:35:13.621989 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:13.621769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:49949)
0808 00:35:13.621892 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:13.621930 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260808 00:35:13.756171 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:13.755937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:52131)
0808 00:35:13.756076 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:13.756118 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260808 00:35:14.009665 22164 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:14.009793 22164 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:14.010133 22164 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
Aug 08 00:35:14 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.195@KRBTEST.COM
Aug 08 00:35:14 dist-test-slave-44d2 krb5kdc[19845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.194@KRBTEST.COM
I20260808 00:35:14.026706 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:14.010296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34309 (local address 127.31.210.193:52645)
0808 00:35:14.010359 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:14.010372 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:14.010470 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:14.011394 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:14.011398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:14.011818 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:14.012017 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:14.012023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.013184 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.013186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:14.014052 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:14.014057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.014743 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.014744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:14.014873 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:14.020641 (+ 5768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:14.020653 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:14.022553 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:14.024800 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:14.024808 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:14.024810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:14.025124 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:14.025413 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:14.025413 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:14.025413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:14.025552 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:14.026070 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:14.026078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:14.026231 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 00:35:14.026353 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 00:35:14.026559 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":83}
I20260808 00:35:14.026736 22166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:14.010476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:52645 (local address 127.31.210.195:34309)
0808 00:35:14.010659 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:14.010672 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:14.010709 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:35:14.011157 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:14.011159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:14.011200 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:14.011313 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:14.012155 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.013086 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.014156 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.014628 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.014772 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:14.022723 (+ 7951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:14.022728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:14.022729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:14.022742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:14.024716 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:14.025210 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:14.025212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:14.025213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:14.025341 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:14.025632 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:14.025634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:14.025634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:14.025986 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:14.026059 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:14.026425 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 00:35:14.026610 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260808 00:35:14.026836 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:14.010531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:41037 (local address 127.31.210.194:46145)
0808 00:35:14.010678 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:14.010686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:14.010712 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:35:14.011215 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:14.011217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:14.011255 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:14.011420 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:14.012159 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.013097 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.014075 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.014624 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.015165 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:14.023156 (+ 7991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:14.023161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:14.023161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:14.023174 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:14.025260 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:14.025628 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:14.025633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:14.025633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:14.025723 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:14.025980 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:14.025981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:14.025981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:14.026283 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:14.026371 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:14.026571 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 00:35:14.026702 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260808 00:35:14.026788 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:14.010346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:46145 (local address 127.31.210.193:41037)
0808 00:35:14.010461 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:14.010472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:14.010541 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:14.011432 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:14.011433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:14.011818 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:14.012017 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:14.012023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.013217 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.013218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:14.013976 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:14.013978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:14.014750 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:14.014751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:14.014873 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:14.020641 (+ 5768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:14.020653 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:14.023016 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:14.025337 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:14.025338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:14.025339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:14.025541 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:14.025797 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:14.025798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:14.025798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:14.025859 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:14.026367 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:14.026368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:14.026415 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:35:14.026491 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 00:35:14.026621 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":48,"thread_start_us":49,"threads_started":1}
W20260808 00:35:14.027964 21616 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
W20260808 00:35:14.028059 21618 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
I20260808 00:35:14.028088 21616 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da51014a186241b4b98612265fc560a9; no voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b
I20260808 00:35:14.028203 22164 raft_consensus.cc:2726] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:35:14.566299 22170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:14.566063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:56125)
0808 00:35:14.566211 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:14.566251 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260808 00:35:14.622781 22173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:14.622546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:46095)
0808 00:35:14.622700 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:14.622740 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260808 00:35:14.756976 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:14.756704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:50637)
0808 00:35:14.756891 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:14.756929 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":113,"threads_started":1}
I20260808 00:35:15.567032 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:15.566795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:33999)
0808 00:35:15.566932 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:15.566979 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260808 00:35:15.623756 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:15.623484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:37839)
0808 00:35:15.623664 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:15.623708 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260808 00:35:15.757720 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:15.757454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:32909)
0808 00:35:15.757636 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:15.757673 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260808 00:35:16.099795 22182 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:16.100052 22182 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:16.100284 22182 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
W20260808 00:35:16.100741 21618 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
W20260808 00:35:16.100859 21616 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
I20260808 00:35:16.100970 21616 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da51014a186241b4b98612265fc560a9; no voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b
I20260808 00:35:16.101176 22182 raft_consensus.cc:2726] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:35:16.567975 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:16.567627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:40105)
0808 00:35:16.567808 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:16.567921 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
I20260808 00:35:16.624469 22185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:16.624252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:47645)
0808 00:35:16.624390 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:16.624424 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260808 00:35:16.758525 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:16.758279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:43979)
0808 00:35:16.758432 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:16.758477 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260808 00:35:17.568943 22187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:17.568656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:46861)
0808 00:35:17.568802 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:17.568853 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260808 00:35:17.625196 22188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:17.624950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:42469)
0808 00:35:17.625114 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:17.625151 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260808 00:35:17.759289 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:17.759025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:38009)
0808 00:35:17.759198 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:17.759241 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260808 00:35:18.569733 22191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:18.569468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:49917)
0808 00:35:18.569634 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:18.569682 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260808 00:35:18.605082 22193 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:18.605191 22193 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:18.605410 22193 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
W20260808 00:35:18.605880 21616 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
W20260808 00:35:18.606010 21618 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
I20260808 00:35:18.606082 21618 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da51014a186241b4b98612265fc560a9; no voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b
I20260808 00:35:18.606185 22193 raft_consensus.cc:2726] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:35:18.626045 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:18.625828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:34201)
0808 00:35:18.625955 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:18.625998 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260808 00:35:18.760046 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:18.759770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:54477)
0808 00:35:18.759953 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:18.759997 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260808 00:35:19.570585 22197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:19.570331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:46721)
0808 00:35:19.570480 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:19.570524 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260808 00:35:19.626848 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:19.626600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:59853)
0808 00:35:19.626755 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:19.626799 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260808 00:35:19.760862 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:19.760618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:55127)
0808 00:35:19.760777 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:19.760814 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260808 00:35:20.571393 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:20.571160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:56083)
0808 00:35:20.571295 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:20.571336 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260808 00:35:20.627627 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:20.627362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:48343)
0808 00:35:20.627532 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:20.627576 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260808 00:35:20.761672 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:20.761437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:36197)
0808 00:35:20.761590 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:20.761625 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260808 00:35:21.572145 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:21.571877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:54177)
0808 00:35:21.572058 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:21.572096 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":66,"threads_started":1}
I20260808 00:35:21.628687 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:21.628300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:58989)
0808 00:35:21.628576 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:21.628613 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260808 00:35:21.762538 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:21.762261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:41265)
0808 00:35:21.762438 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:21.762490 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260808 00:35:22.572973 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:22.572729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:37375)
0808 00:35:22.572877 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:22.572922 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260808 00:35:22.629526 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:22.629190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:36723)
0808 00:35:22.629430 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:22.629479 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":77,"threads_started":1}
I20260808 00:35:22.763347 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:22.763083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:56043)
0808 00:35:22.763250 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:22.763295 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
I20260808 00:35:22.776895 22216 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:22.777015 22216 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:22.777235 22216 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
W20260808 00:35:22.777632 21618 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
W20260808 00:35:22.777748 21616 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
I20260808 00:35:22.777915 21616 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da51014a186241b4b98612265fc560a9; no voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b
I20260808 00:35:22.777999 22216 raft_consensus.cc:2726] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:35:23.573827 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:23.573543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:56809)
0808 00:35:23.573735 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:23.573774 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260808 00:35:23.630364 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:23.630104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:44223)
0808 00:35:23.630270 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:23.630310 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260808 00:35:23.764164 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:23.763932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:60867)
0808 00:35:23.764085 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:23.764118 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260808 00:35:24.574720 22220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:24.574444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:34897)
0808 00:35:24.574628 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:24.574672 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260808 00:35:24.631165 22222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:24.630908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:40915)
0808 00:35:24.631071 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:24.631115 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260808 00:35:24.764987 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:24.764754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:49687)
0808 00:35:24.764893 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:24.764937 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260808 00:35:25.242682 22224 raft_consensus.cc:482] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:25.242787 22224 raft_consensus.cc:504] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4dce29fc0e4ee38e23d049f48eb14a" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34309 } } peers { permanent_uuid: "f432c21575f84cfc9c5cac352c8f482b" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 46145 } } peers { permanent_uuid: "da51014a186241b4b98612265fc560a9" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 42683 } }
I20260808 00:35:25.243029 22224 leader_election.cc:288] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309), f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145)
W20260808 00:35:25.243484 21618 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f432c21575f84cfc9c5cac352c8f482b (127.31.210.194:46145): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
W20260808 00:35:25.243618 21616 leader_election.cc:341] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de4dce29fc0e4ee38e23d049f48eb14a (127.31.210.195:34309): Not found: Tablet not found: fca366819b1945729262f6ccacaf6081
I20260808 00:35:25.243700 21616 leader_election.cc:302] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da51014a186241b4b98612265fc560a9; no voters: de4dce29fc0e4ee38e23d049f48eb14a, f432c21575f84cfc9c5cac352c8f482b
I20260808 00:35:25.243791 22224 raft_consensus.cc:2726] T fca366819b1945729262f6ccacaf6081 P da51014a186241b4b98612265fc560a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:35:25.575611 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:25.575332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:35951)
0808 00:35:25.575521 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:25.575561 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":91,"threads_started":1}
I20260808 00:35:25.632196 22226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:25.631899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:50955)
0808 00:35:25.632095 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:25.632143 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
I20260808 00:35:25.765764 22227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:25.765550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:35975)
0808 00:35:25.765682 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:25.765717 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260808 00:35:26.576465 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:26.576146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:41997)
0808 00:35:26.576330 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:26.576412 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260808 00:35:26.632930 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:26.632707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:46301)
0808 00:35:26.632844 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:26.632883 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260808 00:35:26.766613 22232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:26.766317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:49777)
0808 00:35:26.766521 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:26.766564 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":101,"threads_started":1}
I20260808 00:35:27.577454 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:27.577196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.193:51231)
0808 00:35:27.577364 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:27.577407 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260808 00:35:27.633749 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:27.633522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.194:48653)
0808 00:35:27.633666 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:27.633704 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260808 00:35:27.762384 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 21542
I20260808 00:35:27.769052 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:27.767249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44411 (local address 127.31.210.195:51117)
0808 00:35:27.768924 (+ 1675us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:27.768983 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:44411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":56,"threads_started":1}
I20260808 00:35:27.775717 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 21783
I20260808 00:35:27.785812 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 21967
2026-08-08T00:35:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18952 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:35:27 dist-test-slave-44d2 krb5kdc[22241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:35:27 dist-test-slave-44d2 krb5kdc[22241](info): set up 2 sockets
Aug 08 00:35:27 dist-test-slave-44d2 krb5kdc[22241](info): commencing operation
krb5kdc: starting...
W20260808 00:35:29.049890 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:35:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:35:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:35:29.184639 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46623
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:29.272852 22262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:29.273136 22262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:29.273190 22262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:29.274526 22262 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:35:29.274608 22262 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:35:29.274633 22262 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:35:29.274654 22262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:29.274693 22262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:29.274720 22262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:29.274757 22262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:29.274789 22262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:29.276540 22262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:29.277089 22262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:29.277402 22262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:29.283280 22269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:29.283308 22270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:29.283315 22262 server_base.cc:1034] running on GCE node
W20260808 00:35:29.283280 22273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:29.283766 22262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:29.284051 22262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:29.285224 22262 hybrid_clock.cc:648] HybridClock initialized: now 1786149329285208 us; error 33 us; skew 500 ppm
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:29.287899 22262 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:29.288400 22262 webserver.cc:460] Webserver started at http://127.31.210.254:43139/ using document root <none> and password file <none>
I20260808 00:35:29.288674 22262 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:29.288750 22262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:29.288909 22262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:29.290019 22262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "835ef069a8134a599e4e670c595f9b7c"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "360a3f0c7af8c219b754e4fad92275d4"
server_key_iv: "cdfca2b0c5d7798389e936ea85388210"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.290398 22262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "835ef069a8134a599e4e670c595f9b7c"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "360a3f0c7af8c219b754e4fad92275d4"
server_key_iv: "cdfca2b0c5d7798389e936ea85388210"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.291899 22262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:35:29.292855 22282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.293092 22262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:29.293210 22262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "835ef069a8134a599e4e670c595f9b7c"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "360a3f0c7af8c219b754e4fad92275d4"
server_key_iv: "cdfca2b0c5d7798389e936ea85388210"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.293289 22262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:29.311621 22262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:29.313850 22262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:29.314047 22262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:29.318475 22262 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46623
I20260808 00:35:29.318583 22353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46623 every 8 connection(s)
I20260808 00:35:29.318929 22262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:29.319878 22354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:29.320881 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 22262
I20260808 00:35:29.320990 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:35:29.321224 32587 external_mini_cluster.cc:1428] Setting key 1c20152650d2e8339d7eced0f3085ffe
I20260808 00:35:29.323555 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Bootstrap starting.
I20260808 00:35:29.324206 22354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:29.324550 22354 log.cc:824] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:29.325289 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: No bootstrap required, opened a new log
I20260808 00:35:29.327369 22354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:29.327559 22354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:29.327597 22354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835ef069a8134a599e4e670c595f9b7c, State: Initialized, Role: FOLLOWER
I20260808 00:35:29.327731 22354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:29.327822 22354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:35:29.327857 22354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:35:29.327903 22354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 0 FOLLOWER]: Advancing to term 1
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:29.328711 22354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:29.328822 22354 leader_election.cc:302] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 835ef069a8134a599e4e670c595f9b7c; no voters:
I20260808 00:35:29.329054 22354 leader_election.cc:288] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:35:29.329115 22359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:29.329281 22359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 LEADER]: Becoming Leader. State: Replica: 835ef069a8134a599e4e670c595f9b7c, State: Running, Role: LEADER
I20260808 00:35:29.329358 22354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:29.329434 22359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:29.329814 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "835ef069a8134a599e4e670c595f9b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } } }
I20260808 00:35:29.329950 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:29.330036 22360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 835ef069a8134a599e4e670c595f9b7c. Latest consensus state: current_term: 1 leader_uuid: "835ef069a8134a599e4e670c595f9b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } } }
I20260808 00:35:29.330217 22360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:29.330252 22372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:29.330847 22372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:29.332525 22372 catalog_manager.cc:1347] Generated new cluster ID: 4d83e14ef3484a428fa458f764aaf770
I20260808 00:35:29.332576 22372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:35:29.332933 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.321807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.31.210.254:46623)
0808 00:35:29.322021 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.322030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.322058 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:35:29.322475 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.322483 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.322534 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.322651 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:29.323536 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.324621 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.325407 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.325839 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.325973 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.328682 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:29.328686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:29.328686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:29.328696 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:29.331079 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.331498 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.331501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.331502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.331603 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.331836 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.331837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.331838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.332185 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:29.332346 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.332531 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.332696 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260808 00:35:29.364907 22372 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:35:29.365512 22372 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:35:29.375512 22372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Generated new TSK 0
I20260808 00:35:29.375742 22372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:35:29.412096 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46623
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:29.496457 22387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:29.496690 22387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:29.496713 22387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:29.497888 22387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:29.497931 22387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:29.497964 22387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:35:29.499326 22387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:29.499508 22387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:29.499680 22387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:29.504715 22394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:29.504860 22387 server_base.cc:1034] running on GCE node
W20260808 00:35:29.504735 22398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:29.504985 22393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:29.505239 22387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:29.505514 22387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:29.506706 22387 hybrid_clock.cc:648] HybridClock initialized: now 1786149329506694 us; error 32 us; skew 500 ppm
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:29.509747 22387 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:35:29.510177 22387 webserver.cc:460] Webserver started at http://127.31.210.193:43723/ using document root <none> and password file <none>
I20260808 00:35:29.510445 22387 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:29.510522 22387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:29.510684 22387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:29.511766 22387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "103be76fe1fc414f9e4c56653a4f2278"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "a8b8833b2d51d6aa31a90e9b320e829f"
server_key_iv: "c32e221a89f559e218b84bc89d80d076"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.512140 22387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "103be76fe1fc414f9e4c56653a4f2278"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "a8b8833b2d51d6aa31a90e9b320e829f"
server_key_iv: "c32e221a89f559e218b84bc89d80d076"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.513626 22387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:35:29.514360 22408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.514523 22387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.514632 22387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "103be76fe1fc414f9e4c56653a4f2278"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "a8b8833b2d51d6aa31a90e9b320e829f"
server_key_iv: "c32e221a89f559e218b84bc89d80d076"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.514707 22387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:29.525295 22387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:29.527675 22387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:29.527842 22387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:29.528083 22387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:29.528395 22387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:29.528455 22387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.528542 22387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:29.528576 22387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.534729 22387 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:38087
I20260808 00:35:29.534781 22586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:38087 every 8 connection(s)
I20260808 00:35:29.535096 22387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:35:29.537977 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 22387
I20260808 00:35:29.538098 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:35:29.538291 32587 external_mini_cluster.cc:1428] Setting key 8292a911077bfc801b8324b11824a8b5
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:29.545392 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.535822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:47605 (local address 127.31.210.254:46623)
0808 00:35:29.535917 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.535920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.536107 (+ 187us) server_negotiation.cc:407] Connection header received
0808 00:35:29.536694 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.536695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.536708 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.536771 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:29.537752 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.538349 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.539601 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.539839 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.539921 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.542444 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:29.542446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:29.542446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:29.542464 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:29.543887 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.544323 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.544324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.544325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.544388 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.544692 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.544693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.544693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.544818 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:29.544894 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.545198 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.545299 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:35:29.545673 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.535788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:47605)
0808 00:35:29.535981 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:29.536018 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:29.536589 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:29.536849 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:29.536853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:29.537444 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:29.537646 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.537651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.538455 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.538456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.539506 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.539509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.539929 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.539930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.540063 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.540609 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:29.540618 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:29.542328 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:29.543988 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:29.543997 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:29.543999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:29.544226 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:29.544464 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:29.544465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:29.544465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:29.544619 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:29.544898 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:29.544907 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:29.545054 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 00:35:29.545158 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:35:29.545321 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260808 00:35:29.546154 22587 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46623
I20260808 00:35:29.546253 22587 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:29.546468 22587 heartbeater.cc:511] Master 127.31.210.254:46623 requested a full tablet report, sending...
I20260808 00:35:29.546998 22307 ts_manager.cc:194] Registered new tserver with Master: 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087)
I20260808 00:35:29.547639 22307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:47605
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:35:29.567708 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46623
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:29.651852 22595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:29.652055 22595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:29.652078 22595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:29.653285 22595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:29.653332 22595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:29.653359 22595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:35:29.654855 22595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:29.655103 22595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:29.655335 22595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:29.660377 22606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:29.660383 22604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:29.660383 22603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:29.660609 22595 server_base.cc:1034] running on GCE node
I20260808 00:35:29.660889 22595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:29.661136 22595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:29.662328 22595 hybrid_clock.cc:648] HybridClock initialized: now 1786149329662316 us; error 46 us; skew 500 ppm
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:29.665000 22595 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:35:29.665433 22595 webserver.cc:460] Webserver started at http://127.31.210.194:33369/ using document root <none> and password file <none>
I20260808 00:35:29.665670 22595 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:29.665742 22595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:29.665896 22595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:29.666961 22595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "69746a4ff14245158d14cb5a78399d48"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "ff6f3c4dc52e331d145f15b86212d3c6"
server_key_iv: "363931581c981dbc0e8b99aac9677464"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.667340 22595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "69746a4ff14245158d14cb5a78399d48"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "ff6f3c4dc52e331d145f15b86212d3c6"
server_key_iv: "363931581c981dbc0e8b99aac9677464"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.668747 22595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:35:29.669499 22617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.669682 22595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:35:29.669788 22595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "69746a4ff14245158d14cb5a78399d48"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "ff6f3c4dc52e331d145f15b86212d3c6"
server_key_iv: "363931581c981dbc0e8b99aac9677464"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.669866 22595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:29.693228 22595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:29.695202 22595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:29.695377 22595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:29.695636 22595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:29.695993 22595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:29.696062 22595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.696105 22595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:29.696133 22595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.702081 22595 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:45419
I20260808 00:35:29.702126 22781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:45419 every 8 connection(s)
I20260808 00:35:29.702505 22595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:35:29.703385 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 22595
I20260808 00:35:29.703517 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:35:29.703760 32587 external_mini_cluster.cc:1428] Setting key d5451667ef0419373e753f924838f9ec
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:29.713644 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.703346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:55821 (local address 127.31.210.254:46623)
0808 00:35:29.703406 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.703408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.703665 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:35:29.704259 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.704260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.704270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.704332 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:29.705721 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.706203 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.707326 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.707583 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.707656 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.710382 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:29.710384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:29.710384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:29.710449 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:29.711989 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.712557 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.712558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.712558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.712628 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.712945 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.712946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.712946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.713083 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:29.713141 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.713504 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.713577 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:35:29.713949 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.703267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:55821)
0808 00:35:29.703515 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:29.703567 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:29.704169 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:29.704427 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:29.704431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:29.705322 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:29.705623 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.705630 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.706301 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.706302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.707227 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.707230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.707705 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.707706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.707827 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.708380 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:29.708388 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:29.710250 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:29.712084 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:29.712093 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:29.712096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:29.712439 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:29.712702 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:29.712702 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:29.712703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:29.712869 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:29.713156 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:29.713164 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:29.713351 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 00:35:29.713475 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 00:35:29.713681 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
I20260808 00:35:29.714524 22782 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46623
I20260808 00:35:29.714615 22782 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:29.714767 22782 heartbeater.cc:511] Master 127.31.210.254:46623 requested a full tablet report, sending...
I20260808 00:35:29.715096 22307 ts_manager.cc:194] Registered new tserver with Master: 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419)
I20260808 00:35:29.715534 22307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:55821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:35:29.735944 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46623
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:29.820919 22790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:29.821156 22790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:29.821182 22790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:29.822294 22790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:29.822331 22790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:29.822358 22790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:35:29.823787 22790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:29.824007 22790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:29.824194 22790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:29.829457 22796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:29.829447 22801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:29.829447 22798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:29.829703 22790 server_base.cc:1034] running on GCE node
I20260808 00:35:29.829919 22790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:29.830252 22790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:29.831408 22790 hybrid_clock.cc:648] HybridClock initialized: now 1786149329831382 us; error 45 us; skew 500 ppm
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:29.834337 22790 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:35:29.834811 22790 webserver.cc:460] Webserver started at http://127.31.210.195:36249/ using document root <none> and password file <none>
I20260808 00:35:29.835016 22790 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:29.835057 22790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:29.835148 22790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:35:29.836140 22790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "b2f343278d3b4733adc49886f6537903"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "5037ca68b180e383af2c9ad5c918185b"
server_key_iv: "b17086bcdb4672a467586f8f4084fd16"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.836440 22790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "b2f343278d3b4733adc49886f6537903"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "5037ca68b180e383af2c9ad5c918185b"
server_key_iv: "b17086bcdb4672a467586f8f4084fd16"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.837790 22790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:29.838511 22810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.838696 22790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:35:29.838817 22790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "b2f343278d3b4733adc49886f6537903"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "5037ca68b180e383af2c9ad5c918185b"
server_key_iv: "b17086bcdb4672a467586f8f4084fd16"
server_key_version: "encryptionkey@0"
I20260808 00:35:29.838943 22790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:29.857810 22790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:29.860030 22790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:29.860160 22790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:29.860396 22790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:29.860795 22790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:29.860838 22790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.860862 22790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:29.860882 22790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:29.867319 22790 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:41261
I20260808 00:35:29.867430 22977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:41261 every 8 connection(s)
I20260808 00:35:29.867828 22790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:35:29.872092 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 22790
I20260808 00:35:29.872241 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:35:29.872547 32587 external_mini_cluster.cc:1428] Setting key 7a1de0429baac9a98506b0ffe3323271
Aug 08 00:35:29 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149329, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:29.878124 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.868744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:42659 (local address 127.31.210.254:46623)
0808 00:35:29.868824 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.868826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.869017 (+ 191us) server_negotiation.cc:407] Connection header received
0808 00:35:29.869493 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.869494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.869507 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.869586 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:29.870548 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.871218 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.872327 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.872589 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.872659 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.875251 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:29.875252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:29.875252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:29.875259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:29.876835 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.877244 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.877245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.877245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.877300 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.877521 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.877522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.877522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.877625 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:29.877677 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.877970 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.878061 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:35:29.878261 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.868697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:42659)
0808 00:35:29.868871 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:29.868915 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:29.869401 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:29.869642 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:29.869645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:29.870235 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:29.870435 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.870440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.871329 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.871330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.872202 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.872205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.872789 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.872790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.872942 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.873594 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:29.873604 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:29.875155 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:29.876938 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:29.876944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:29.876946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:29.877170 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:29.877363 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:29.877364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:29.877364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:29.877465 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:29.877700 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:29.877706 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:29.877829 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 00:35:29.877925 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 00:35:29.878065 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260808 00:35:29.878809 22978 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46623
I20260808 00:35:29.878895 22978 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:29.879155 22978 heartbeater.cc:511] Master 127.31.210.254:46623 requested a full tablet report, sending...
I20260808 00:35:29.879592 22307 ts_manager.cc:194] Registered new tserver with Master: b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261)
I20260808 00:35:29.880061 22307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:42659
I20260808 00:35:29.884845 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:35:29.892267 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.886322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55328 (local address 127.31.210.254:46623)
0808 00:35:29.886369 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.886371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.886468 (+ 97us) server_negotiation.cc:407] Connection header received
0808 00:35:29.886522 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.886522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.886533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.886593 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:29.887173 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.887747 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.888552 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.888752 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.888814 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.889981 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:29.889983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:29.889983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:29.889989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:29.891401 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.891685 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.891685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.891685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.891726 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:29.891866 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:29.891866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:29.891866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:29.891993 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:29.892061 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.892154 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.892208 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:35:29.893966 22307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:35:29.895567 22307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:35:29.902194 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.898068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:45419 (local address 127.0.0.1:53826)
0808 00:35:29.898447 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:29.898457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:29.898514 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:29.898776 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:29.898776 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:29.898778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:29.898819 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.898820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.900189 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.900190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.901147 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.901150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.901862 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.901863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.901943 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.901956 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 00:35:29.902061 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 00:35:29.902116 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260808 00:35:29.902205 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.898341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.31.210.194:45419)
0808 00:35:29.898547 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.898551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.898560 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:35:29.898601 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.898603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.898618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.898690 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:29.898898 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.900044 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.901234 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.901768 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.901864 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.901925 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.902081 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.902126 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260808 00:35:29.902403 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.898067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38087 (local address 127.0.0.1:32958)
0808 00:35:29.898326 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:29.898337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:29.898396 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:29.898926 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:29.898927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:29.898929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:29.898985 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.898986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.899869 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.899870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.901216 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.901219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.902161 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.902163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.902228 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.902231 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:35:29.902261 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 00:35:29.902316 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":23,"thread_start_us":103,"threads_started":1}
I20260808 00:35:29.902642 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.898021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:41261 (local address 127.0.0.1:45254)
0808 00:35:29.898222 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:29.898262 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:29.898338 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:29.898614 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:29.898617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:29.898619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:29.898808 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.898810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.900156 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.900157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.901309 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:29.901311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.902217 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.902218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:29.902399 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.902402 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:35:29.902537 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 00:35:29.902578 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":29,"thread_start_us":58,"threads_started":1}
I20260808 00:35:29.902765 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.898565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.31.210.193:38087)
0808 00:35:29.898744 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.898749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.898762 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:35:29.898803 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.898805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.898821 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.899061 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:29.899118 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.899769 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.901325 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.902056 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.902529 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.902594 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.902621 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.902664 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260808 00:35:29.903025 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:29.898098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.31.210.195:41261)
0808 00:35:29.898367 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:29.898372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:29.898382 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:35:29.898423 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:29.898426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:29.898442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:29.898520 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:29.898971 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.900044 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.901392 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:29.902056 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:29.902134 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:29.902189 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:29.902898 (+ 709us) server_negotiation.cc:299] Negotiation successful
0808 00:35:29.902945 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260808 00:35:29.902967 22689 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddc86d5d4744426a37ea5e1d9b23971 (DEFAULT_TABLE table=test-table [id=2354bb6b1e124506956a4eeba178540f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:29.903363 22689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddc86d5d4744426a37ea5e1d9b23971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:29.903430 22488 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddc86d5d4744426a37ea5e1d9b23971 (DEFAULT_TABLE table=test-table [id=2354bb6b1e124506956a4eeba178540f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:29.903759 22488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddc86d5d4744426a37ea5e1d9b23971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:29.903698 22881 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddc86d5d4744426a37ea5e1d9b23971 (DEFAULT_TABLE table=test-table [id=2354bb6b1e124506956a4eeba178540f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:29.904008 22881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddc86d5d4744426a37ea5e1d9b23971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:29.906608 23000 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Bootstrap starting.
I20260808 00:35:29.907194 23002 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap starting.
I20260808 00:35:29.907428 23000 tablet_bootstrap.cc:654] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:29.907861 23002 tablet_bootstrap.cc:654] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:29.907914 23000 log.cc:824] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:29.908524 23002 log.cc:824] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:29.908625 23001 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Bootstrap starting.
I20260808 00:35:29.908844 23000 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: No bootstrap required, opened a new log
I20260808 00:35:29.908931 23000 ts_tablet_manager.cc:1397] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:29.909284 23001 tablet_bootstrap.cc:654] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:29.909322 23002 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: No bootstrap required, opened a new log
I20260808 00:35:29.909482 23002 ts_tablet_manager.cc:1397] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:35:29.909902 23001 log.cc:824] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:29.910735 23001 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: No bootstrap required, opened a new log
I20260808 00:35:29.910805 23001 ts_tablet_manager.cc:1397] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:29.910908 23000 raft_consensus.cc:348] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:29.911088 23000 raft_consensus.cc:374] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:29.911132 23000 raft_consensus.cc:729] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f343278d3b4733adc49886f6537903, State: Initialized, Role: FOLLOWER
I20260808 00:35:29.911250 23000 consensus_queue.cc:260] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:29.911527 23000 ts_tablet_manager.cc:1428] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:35:29.911535 23002 raft_consensus.cc:348] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:29.911675 22978 heartbeater.cc:503] Master 127.31.210.254:46623 was elected leader, sending a full tablet report...
I20260808 00:35:29.911710 23002 raft_consensus.cc:374] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:29.911753 23002 raft_consensus.cc:729] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103be76fe1fc414f9e4c56653a4f2278, State: Initialized, Role: FOLLOWER
I20260808 00:35:29.911861 23002 consensus_queue.cc:260] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:29.912133 23002 ts_tablet_manager.cc:1428] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:35:29.912158 22587 heartbeater.cc:503] Master 127.31.210.254:46623 was elected leader, sending a full tablet report...
I20260808 00:35:29.913244 23001 raft_consensus.cc:348] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:29.913424 23001 raft_consensus.cc:374] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:29.913477 23001 raft_consensus.cc:729] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69746a4ff14245158d14cb5a78399d48, State: Initialized, Role: FOLLOWER
I20260808 00:35:29.913589 23001 consensus_queue.cc:260] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:29.913849 23001 ts_tablet_manager.cc:1428] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:35:29.913882 22782 heartbeater.cc:503] Master 127.31.210.254:46623 was elected leader, sending a full tablet report...
W20260808 00:35:29.953125 22783 tablet.cc:2369] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:35:30.035851 22588 tablet.cc:2369] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:35:30.042202 23012 raft_consensus.cc:482] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:30.042385 23012 raft_consensus.cc:504] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:30.042870 23012 leader_election.cc:288] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087)
I20260808 00:35:30.046069 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:30.043267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:44633 (local address 127.31.210.195:41261)
0808 00:35:30.043313 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:30.043314 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:30.043324 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:35:30.043360 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:30.043361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:30.043372 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:30.043436 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:30.043637 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.044175 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.045231 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.045674 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.045734 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:30.045743 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:30.045948 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 00:35:30.045995 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:35:30.046137 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:30.043085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:41261 (local address 127.31.210.194:44633)
0808 00:35:30.043170 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:30.043183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:30.043250 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:30.043464 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:30.043465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:30.043468 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:30.043552 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:30.043554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.044273 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.044274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:30.045121 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:30.045124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.045760 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.045762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:30.045852 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:30.045856 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:35:30.045937 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 00:35:30.046033 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 00:35:30.046137 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:30.043174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:54743 (local address 127.31.210.193:38087)
0808 00:35:30.043243 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:30.043246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:30.043469 (+ 223us) server_negotiation.cc:407] Connection header received
0808 00:35:30.043503 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:30.043504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:30.043515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:30.043569 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:30.043798 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.044311 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.045338 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.045745 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.045822 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:30.045833 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:30.046027 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 00:35:30.046068 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:35:30.046293 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:30.043085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38087 (local address 127.31.210.194:54743)
0808 00:35:30.043331 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:30.043344 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:30.043425 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:30.043604 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:30.043605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:30.043607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:30.043687 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:30.043689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.044401 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.044402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:30.045233 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:30.045235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.045880 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.045882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:30.045950 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:30.045955 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:30.046012 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:35:30.046053 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":28,"thread_start_us":139,"threads_started":1}
I20260808 00:35:30.046511 22911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddc86d5d4744426a37ea5e1d9b23971" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f343278d3b4733adc49886f6537903" is_pre_election: true
I20260808 00:35:30.046609 22521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddc86d5d4744426a37ea5e1d9b23971" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103be76fe1fc414f9e4c56653a4f2278" is_pre_election: true
I20260808 00:35:30.046695 22911 raft_consensus.cc:2454] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 0.
I20260808 00:35:30.046756 22521 raft_consensus.cc:2454] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 0.
I20260808 00:35:30.046969 22621 leader_election.cc:302] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69746a4ff14245158d14cb5a78399d48, b2f343278d3b4733adc49886f6537903; no voters:
I20260808 00:35:30.047143 23012 raft_consensus.cc:2781] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:35:30.047210 23012 raft_consensus.cc:482] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:35:30.047228 23012 raft_consensus.cc:3037] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:30.047937 23012 raft_consensus.cc:504] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:30.048091 23012 leader_election.cc:288] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 election: Requested vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087)
I20260808 00:35:30.048241 22911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddc86d5d4744426a37ea5e1d9b23971" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f343278d3b4733adc49886f6537903"
I20260808 00:35:30.048265 22521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddc86d5d4744426a37ea5e1d9b23971" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103be76fe1fc414f9e4c56653a4f2278"
I20260808 00:35:30.048318 22911 raft_consensus.cc:3037] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:30.048379 22521 raft_consensus.cc:3037] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:30.049156 22521 raft_consensus.cc:2454] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 1.
I20260808 00:35:30.049158 22911 raft_consensus.cc:2454] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 1.
I20260808 00:35:30.049358 22622 leader_election.cc:302] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103be76fe1fc414f9e4c56653a4f2278, 69746a4ff14245158d14cb5a78399d48; no voters:
I20260808 00:35:30.049477 23012 raft_consensus.cc:2781] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:30.049638 23012 raft_consensus.cc:686] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 1 LEADER]: Becoming Leader. State: Replica: 69746a4ff14245158d14cb5a78399d48, State: Running, Role: LEADER
I20260808 00:35:30.049744 23012 consensus_queue.cc:237] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:30.050665 22307 catalog_manager.cc:5495] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69746a4ff14245158d14cb5a78399d48 (127.31.210.194). New cstate: current_term: 1 leader_uuid: "69746a4ff14245158d14cb5a78399d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } health_report { overall_health: UNKNOWN } } }
I20260808 00:35:30.087649 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:30.085124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.31.210.194:45419)
0808 00:35:30.085213 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:30.085215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:30.085223 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:35:30.085265 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:30.085266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:30.085285 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:30.085350 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:35:30.085564 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.086121 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.086889 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:30.087106 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:30.087161 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:30.087297 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:35:30.087400 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:35:30.087464 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:30.087514 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 00:35:30.087532 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:35:30.089656 22521 raft_consensus.cc:1261] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Refusing update from remote peer 69746a4ff14245158d14cb5a78399d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:30.089704 22911 raft_consensus.cc:1261] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Refusing update from remote peer 69746a4ff14245158d14cb5a78399d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:30.090095 23014 consensus_queue.cc:1035] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:30.090240 23012 consensus_queue.cc:1035] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:30.093173 23021 mvcc.cc:205] Tried to move back new op lower bound from 7316067656044417024 to 7316067655884476416. Current Snapshot: MvccSnapshot[applied={T|T < 7316067656044417024}]
I20260808 00:35:30.093822 23018 mvcc.cc:205] Tried to move back new op lower bound from 7316067656044417024 to 7316067655884476416. Current Snapshot: MvccSnapshot[applied={T|T < 7316067656044417024}]
I20260808 00:35:30.094949 22307 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:30.095037 22307 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:30.095829 22307 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Sending DeleteTablet for 3 replicas of tablet 5ddc86d5d4744426a37ea5e1d9b23971
I20260808 00:35:30.096174 22689 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddc86d5d4744426a37ea5e1d9b23971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:53826
I20260808 00:35:30.096174 22881 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddc86d5d4744426a37ea5e1d9b23971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:45254
I20260808 00:35:30.096414 22488 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddc86d5d4744426a37ea5e1d9b23971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:32958
I20260808 00:35:30.096495 23036 tablet_replica.cc:330] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: stopping tablet replica
I20260808 00:35:30.096596 23036 raft_consensus.cc:2229] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:35:30.096683 23034 tablet_replica.cc:330] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: stopping tablet replica
I20260808 00:35:30.096724 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 22387
I20260808 00:35:30.096733 23036 raft_consensus.cc:2258] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:30.096875 23034 raft_consensus.cc:2229] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:30.097052 23034 raft_consensus.cc:2258] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:30.102131 23034 ts_tablet_manager.cc:1905] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:30.104413 23036 ts_tablet_manager.cc:1905] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:30.105641 23034 ts_tablet_manager.cc:1918] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:30.105783 23034 log.cc:1196] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/5ddc86d5d4744426a37ea5e1d9b23971
I20260808 00:35:30.105801 23036 ts_tablet_manager.cc:1918] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:30.105872 23036 log.cc:1196] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/5ddc86d5d4744426a37ea5e1d9b23971
I20260808 00:35:30.106132 23034 ts_tablet_manager.cc:1939] T 5ddc86d5d4744426a37ea5e1d9b23971 P b2f343278d3b4733adc49886f6537903: Deleting consensus metadata
I20260808 00:35:30.106149 23036 ts_tablet_manager.cc:1939] T 5ddc86d5d4744426a37ea5e1d9b23971 P 69746a4ff14245158d14cb5a78399d48: Deleting consensus metadata
I20260808 00:35:30.106570 22285 catalog_manager.cc:4841] TS b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261): tablet 5ddc86d5d4744426a37ea5e1d9b23971 (table test-table [id=2354bb6b1e124506956a4eeba178540f]) successfully deleted
I20260808 00:35:30.106634 22284 catalog_manager.cc:4841] TS 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419): tablet 5ddc86d5d4744426a37ea5e1d9b23971 (table test-table [id=2354bb6b1e124506956a4eeba178540f]) successfully deleted
W20260808 00:35:30.109436 22287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:38087 (error 108)
I20260808 00:35:30.109987 22996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:30.109843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38087 (local address 127.0.0.1:32962)
0808 00:35:30.109909 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:30.109946 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:35:30.110123 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 22595
W20260808 00:35:30.118846 22287 catalog_manager.cc:4584] TS 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ddc86d5d4744426a37ea5e1d9b23971: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38087: connect: Connection refused (error 111)
I20260808 00:35:30.123631 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 22790
I20260808 00:35:30.132839 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 22262
I20260808 00:35:50.146492 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46623
--webserver_interface=127.31.210.254
--webserver_port=43139
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:35:50.233428 23042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:50.233642 23042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:50.233665 23042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:50.234993 23042 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:35:50.235049 23042 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:35:50.235062 23042 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:35:50.235074 23042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:35:50.235110 23042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:50.235141 23042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:50.235158 23042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:35:50.235181 23042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:35:50.236783 23042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=43139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:50.237071 23042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:50.237340 23042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:50.247618 23049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:50.247648 23051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:50.247835 23042 server_base.cc:1034] running on GCE node
W20260808 00:35:50.247761 23053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:50.248167 23042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:50.248564 23042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:50.249745 23042 hybrid_clock.cc:648] HybridClock initialized: now 1786149350249732 us; error 30 us; skew 500 ppm
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:50.252571 23042 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:35:50.253046 23042 webserver.cc:460] Webserver started at http://127.31.210.254:43139/ using document root <none> and password file <none>
I20260808 00:35:50.253307 23042 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:50.253401 23042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:50.254840 23042 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:35:50.255540 23064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.255721 23042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:35:50.255834 23042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "835ef069a8134a599e4e670c595f9b7c"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "360a3f0c7af8c219b754e4fad92275d4"
server_key_iv: "cdfca2b0c5d7798389e936ea85388210"
server_key_version: "encryptionkey@0"
I20260808 00:35:50.256291 23042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:50.264966 23042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:50.267453 23042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:50.267632 23042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:50.271911 23042 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46623
I20260808 00:35:50.271947 23135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46623 every 8 connection(s)
I20260808 00:35:50.272379 23042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:35:50.273402 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 23042
I20260808 00:35:50.273772 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:38087
--local_ip_for_outbound_sockets=127.31.210.193
--tserver_master_addrs=127.31.210.254:46623
--webserver_port=43723
--webserver_interface=127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:50.274603 23137 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:35:50.275751 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Bootstrap starting.
I20260808 00:35:50.284690 23137 log.cc:824] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:50.287040 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:35:50.287374 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Bootstrap complete.
I20260808 00:35:50.289872 23137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:50.290119 23137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835ef069a8134a599e4e670c595f9b7c, State: Initialized, Role: FOLLOWER
I20260808 00:35:50.290299 23137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:50.290361 23137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:35:50.290416 23137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:35:50.290467 23137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:35:50.291538 23137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:50.291657 23137 leader_election.cc:302] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 835ef069a8134a599e4e670c595f9b7c; no voters:
I20260808 00:35:50.291873 23137 leader_election.cc:288] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:35:50.291910 23142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:35:50.292038 23142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 LEADER]: Becoming Leader. State: Replica: 835ef069a8134a599e4e670c595f9b7c, State: Running, Role: LEADER
I20260808 00:35:50.292178 23137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:35:50.292217 23142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:35:50.292644 23144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 835ef069a8134a599e4e670c595f9b7c. Latest consensus state: current_term: 2 leader_uuid: "835ef069a8134a599e4e670c595f9b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } } }
I20260808 00:35:50.292656 23143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "835ef069a8134a599e4e670c595f9b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } } }
I20260808 00:35:50.292744 23144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:50.292773 23143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: This master's current role is: LEADER
I20260808 00:35:50.293418 23150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:35:50.294205 23150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:35:50.294304 23150 catalog_manager.cc:1259] Loaded cluster ID: 4d83e14ef3484a428fa458f764aaf770
I20260808 00:35:50.294327 23150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:35:50.295497 23150 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:35:50.295679 23150 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Loaded TSK: 0
I20260808 00:35:50.304584 23150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Generated new TSK 1
I20260808 00:35:50.304692 23150 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:35:50.364876 23140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:50.365094 23140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:50.365118 23140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:50.366293 23140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:50.366338 23140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:50.366365 23140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:35:50.367834 23140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:38087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=43723
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:50.368080 23140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:50.368347 23140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:50.373468 23178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:50.373479 23183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:50.373574 23179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:50.373718 23140 server_base.cc:1034] running on GCE node
I20260808 00:35:50.374073 23140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:50.374362 23140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:50.375555 23140 hybrid_clock.cc:648] HybridClock initialized: now 1786149350375541 us; error 29 us; skew 500 ppm
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:50.378667 23140 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:35:50.379179 23140 webserver.cc:460] Webserver started at http://127.31.210.193:43723/ using document root <none> and password file <none>
I20260808 00:35:50.379411 23140 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:50.379534 23140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:50.381050 23140 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:35:50.381811 23198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.382110 23140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:35:50.382225 23140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "103be76fe1fc414f9e4c56653a4f2278"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "a8b8833b2d51d6aa31a90e9b320e829f"
server_key_iv: "c32e221a89f559e218b84bc89d80d076"
server_key_version: "encryptionkey@0"
I20260808 00:35:50.382591 23140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:50.401460 23140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:50.403983 23140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:50.404160 23140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:50.404424 23140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:50.405030 23213 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:35:50.406600 23140 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:35:50.406657 23140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:35:50.406692 23140 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:35:50.407380 23140 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:35:50.407428 23140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:35:50.407472 23213 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap starting.
I20260808 00:35:50.415405 23140 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:38087
I20260808 00:35:50.415985 23140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:35:50.419759 23213 log.cc:824] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:50.419876 23369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:38087 every 8 connection(s)
I20260808 00:35:50.420624 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 23140
I20260808 00:35:50.420936 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:45419
--local_ip_for_outbound_sockets=127.31.210.194
--tserver_master_addrs=127.31.210.254:46623
--webserver_port=33369
--webserver_interface=127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:50.422832 23213 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:35:50.423151 23213 tablet_bootstrap.cc:492] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap complete.
I20260808 00:35:50.424614 23213 ts_tablet_manager.cc:1397] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.004s
I20260808 00:35:50.426400 23213 raft_consensus.cc:348] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.426877 23213 raft_consensus.cc:729] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103be76fe1fc414f9e4c56653a4f2278, State: Initialized, Role: FOLLOWER
I20260808 00:35:50.427062 23213 consensus_queue.cc:260] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:50.427435 23213 ts_tablet_manager.cc:1428] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 00:35:50.432231 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.416837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:46417)
0808 00:35:50.417058 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.417106 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.417684 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.420741 (+ 3057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.420746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.421208 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:50.421495 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.421503 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.423231 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.423232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.424499 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.424503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.425291 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.425292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.425443 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.426097 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:50.426107 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:50.427839 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:50.430451 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.430459 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.430461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.430775 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.431091 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.431092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.431092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.431239 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.431755 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:50.431759 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:50.431869 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.431963 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.432087 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":135,"threads_started":1}
I20260808 00:35:50.432464 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.417005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:46417 (local address 127.31.210.254:46623)
0808 00:35:50.420173 (+ 3168us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.420184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.420231 (+ 47us) server_negotiation.cc:407] Connection header received
0808 00:35:50.420368 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.420397 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.420454 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.421639 (+ 1185us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:50.421727 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.423091 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.424608 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.425180 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.425737 (+ 557us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.427963 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:50.427968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:50.427969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:50.427982 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:50.430348 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.430871 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.430875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.430875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.431002 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.431321 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.431322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.431323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.431672 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:50.431784 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.432060 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.432220 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3110,"thread_start_us":91,"threads_started":1}
I20260808 00:35:50.433334 23370 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46623
I20260808 00:35:50.433423 23370 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:50.433621 23370 heartbeater.cc:511] Master 127.31.210.254:46623 requested a full tablet report, sending...
I20260808 00:35:50.434525 23088 ts_manager.cc:194] Registered new tserver with Master: 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087)
I20260808 00:35:50.435822 23088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:46417
I20260808 00:35:50.436829 23370 heartbeater.cc:503] Master 127.31.210.254:46623 was elected leader, sending a full tablet report...
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.193@KRBTEST.COM
I20260808 00:35:50.443688 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.435176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38087 (local address 127.0.0.1:52654)
0808 00:35:50.435363 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.435404 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.435476 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.435701 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.435708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.435979 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:50.436024 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.436026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.437284 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.437285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.438198 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.438200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.438655 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.438656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.438734 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.439157 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:50.439162 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:50.440647 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:50.442367 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.442375 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.442377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.442603 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.442841 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.442842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.442842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.442911 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.443301 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:50.443308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:50.443361 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.443456 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.443585 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
I20260808 00:35:50.443697 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.435310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.31.210.193:38087)
0808 00:35:50.435486 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.435490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.435501 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:35:50.435582 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.435584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.435602 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.435691 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:50.436265 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.437187 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.438284 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.438578 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.438636 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.440733 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:50.440741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:50.440741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:50.440750 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:50.442232 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.442698 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.442701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.442702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.442766 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.443066 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.443067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.443067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.443216 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:50.443308 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.443518 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.443611 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260808 00:35:50.444147 23274 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddc86d5d4744426a37ea5e1d9b23971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:30 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52654
I20260808 00:35:50.444787 23386 tablet_replica.cc:330] stopping tablet replica
I20260808 00:35:50.444933 23386 raft_consensus.cc:2229] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:50.445019 23386 raft_consensus.cc:2258] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:50.445323 23386 ts_tablet_manager.cc:1905] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:50.447809 23386 ts_tablet_manager.cc:1918] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:50.447906 23386 log.cc:1196] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal/wals/5ddc86d5d4744426a37ea5e1d9b23971
I20260808 00:35:50.448154 23386 ts_tablet_manager.cc:1939] T 5ddc86d5d4744426a37ea5e1d9b23971 P 103be76fe1fc414f9e4c56653a4f2278: Deleting consensus metadata
I20260808 00:35:50.448733 23069 catalog_manager.cc:4841] TS 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087): tablet 5ddc86d5d4744426a37ea5e1d9b23971 (table test-table [id=2354bb6b1e124506956a4eeba178540f]) successfully deleted
W20260808 00:35:50.512645 23376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:50.512849 23376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:50.512872 23376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:50.514165 23376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:50.514213 23376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:50.514240 23376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:35:50.515801 23376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:45419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=33369
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:50.516072 23376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:50.516335 23376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:50.521915 23393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:50.521938 23395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:50.521943 23397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:50.522019 23376 server_base.cc:1034] running on GCE node
I20260808 00:35:50.522377 23376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:50.522634 23376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:50.523818 23376 hybrid_clock.cc:648] HybridClock initialized: now 1786149350523793 us; error 36 us; skew 500 ppm
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:50.526393 23376 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:35:50.526880 23376 webserver.cc:460] Webserver started at http://127.31.210.194:33369/ using document root <none> and password file <none>
I20260808 00:35:50.527139 23376 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:50.527251 23376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:50.528769 23376 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:50.529433 23407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.529618 23376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:50.529727 23376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "69746a4ff14245158d14cb5a78399d48"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "ff6f3c4dc52e331d145f15b86212d3c6"
server_key_iv: "363931581c981dbc0e8b99aac9677464"
server_key_version: "encryptionkey@0"
I20260808 00:35:50.530102 23376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:50.539644 23376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:50.541924 23376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:50.542105 23376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:50.542342 23376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:50.542663 23376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:50.542719 23376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.542760 23376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:50.542788 23376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.549091 23376 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:45419
I20260808 00:35:50.549154 23572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:45419 every 8 connection(s)
I20260808 00:35:50.549576 23376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:50.558964 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 23376
I20260808 00:35:50.559327 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:41261
--local_ip_for_outbound_sockets=127.31.210.195
--tserver_master_addrs=127.31.210.254:46623
--webserver_port=36249
--webserver_interface=127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:35:50.559573 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.550338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:58053 (local address 127.31.210.254:46623)
0808 00:35:50.550386 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.550388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.550670 (+ 282us) server_negotiation.cc:407] Connection header received
0808 00:35:50.551232 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.551233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.551246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.551293 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:50.552299 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.552899 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.553947 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.554175 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.554265 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.556563 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:50.556564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:50.556565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:50.556572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:50.557901 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.558294 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.558294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.558295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.558349 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.558685 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.558686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.558686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.558846 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:50.558917 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.559331 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.559439 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:35:50.559717 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.550342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:58053)
0808 00:35:50.550533 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.550570 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.551109 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.551376 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.551382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.551986 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:50.552187 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.552192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.553005 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.553007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.553833 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.553835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.554280 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.554281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.554421 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.554918 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:50.554927 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:50.556398 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:50.557993 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.558000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.558002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.558226 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.558462 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.558463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.558464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.558619 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.558916 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:50.558922 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:50.559149 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.559244 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.559466 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260808 00:35:50.560331 23574 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46623
I20260808 00:35:50.560452 23574 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:50.560709 23574 heartbeater.cc:511] Master 127.31.210.254:46623 requested a full tablet report, sending...
I20260808 00:35:50.561196 23088 ts_manager.cc:194] Registered new tserver with Master: 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419)
I20260808 00:35:50.561623 23088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:58053
W20260808 00:35:50.646227 23581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:35:50.646425 23581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:35:50.646447 23581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:35:50.647610 23581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:35:50.647653 23581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:35:50.647680 23581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:35:50.649148 23581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:41261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=36249
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:35:50.649441 23581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:35:50.649789 23581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:35:50.655953 23591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:50.655982 23593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:35:50.656029 23589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:35:50.656185 23581 server_base.cc:1034] running on GCE node
I20260808 00:35:50.656404 23581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:35:50.656694 23581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:35:50.657876 23581 hybrid_clock.cc:648] HybridClock initialized: now 1786149350657860 us; error 30 us; skew 500 ppm
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:35:50.660713 23581 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:35:50.661188 23581 webserver.cc:460] Webserver started at http://127.31.210.195:36249/ using document root <none> and password file <none>
I20260808 00:35:50.661377 23581 fs_manager.cc:359] Metadata directory not provided
I20260808 00:35:50.661428 23581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:35:50.662715 23581 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:35:50.663269 23607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.663457 23581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:35:50.663566 23581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "b2f343278d3b4733adc49886f6537903"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "5037ca68b180e383af2c9ad5c918185b"
server_key_iv: "b17086bcdb4672a467586f8f4084fd16"
server_key_version: "encryptionkey@0"
I20260808 00:35:50.663915 23581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:35:50.682176 23581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:35:50.684690 23581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:35:50.684875 23581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:35:50.685175 23581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:35:50.685503 23581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:35:50.685559 23581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.685595 23581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:35:50.685626 23581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:35:50.691906 23581 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:41261
I20260808 00:35:50.691974 23767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:41261 every 8 connection(s)
I20260808 00:35:50.692406 23581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:35:50.695941 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 23581
Aug 08 00:35:50 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149350, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:35:50.703177 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.693231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:58211 (local address 127.31.210.254:46623)
0808 00:35:50.693284 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.693286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.693573 (+ 287us) server_negotiation.cc:407] Connection header received
0808 00:35:50.694051 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.694053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.694063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.694118 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:50.695045 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.695529 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.696979 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.697318 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.697447 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.699886 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:50.699888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:50.699888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:50.699896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:50.701638 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.702147 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.702149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.702149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.702205 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.702508 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.702508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.702509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.702609 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:50.702698 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.702988 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.703081 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:35:50.703410 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.693244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:58211)
0808 00:35:50.693432 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.693482 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.693957 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.694175 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.694178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.694756 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:35:50.694957 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.694962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.695640 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.695642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.696886 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.696889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.697447 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.697448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.697587 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.698197 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:35:50.698208 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:35:50.699799 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:35:50.701717 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.701724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.701726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.702061 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.702279 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:35:50.702280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:35:50.702280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:35:50.702427 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:35:50.702690 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:35:50.702698 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:35:50.702845 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.702964 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.703141 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260808 00:35:50.703835 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.696458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.31.210.254:46623)
0808 00:35:50.696608 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.696610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.696653 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:35:50.696734 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.696735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.696746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.696810 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:50.697609 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.698288 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.699102 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.699421 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.699503 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.700845 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:50.700846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:50.700847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:50.700854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:50.702565 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.702916 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.702917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.702918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.702986 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.703202 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.703202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.703203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.703348 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:50.703543 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.703630 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.703712 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260808 00:35:50.703871 23768 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46623
I20260808 00:35:50.703967 23768 heartbeater.cc:464] Registering TS with master...
I20260808 00:35:50.704156 23768 heartbeater.cc:511] Master 127.31.210.254:46623 requested a full tablet report, sending...
I20260808 00:35:50.704608 23088 ts_manager.cc:194] Registered new tserver with Master: b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261)
I20260808 00:35:50.704934 23088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:58211
I20260808 00:35:50.715560 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:35:50.718634 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.715931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.31.210.254:46623)
0808 00:35:50.716007 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.716009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.716182 (+ 173us) server_negotiation.cc:407] Connection header received
0808 00:35:50.716290 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.716291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.716326 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.716409 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:35:50.716577 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.717057 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.717837 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.718045 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.718134 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.718321 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:35:50.718368 (+ 47us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 00:35:50.718557 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59052: token expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 00:35:50.718726 23774 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59052: token expired
W20260808 00:35:50.718823 22988 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.210.254:46623)
I20260808 00:35:50.725090 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.719150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.31.210.254:46623)
0808 00:35:50.719185 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.719187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.719194 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:35:50.719253 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.719254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.719263 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.719317 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:35:50.719846 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.720308 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.721097 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.721295 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.721372 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.722624 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:35:50.722625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:35:50.722625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:35:50.722633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:35:50.723887 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.724255 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.724257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.724257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.724319 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:35:50.724565 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:35:50.724566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:35:50.724567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:35:50.724681 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:35:50.724792 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.724898 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.724989 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 00:35:50.726285 23088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:35:50.727890 23088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:35:50.730372 23274 tablet_service.cc:1467] Processing CreateTablet for tablet ff0b9358b14042a5b3d96964031a9ad8 (DEFAULT_TABLE table=test-table [id=b9f035227161421781d65dee6b9a60fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:50.730630 23274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff0b9358b14042a5b3d96964031a9ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:50.732299 23213 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap starting.
I20260808 00:35:50.732685 23213 tablet_bootstrap.cc:654] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:50.733316 23213 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: No bootstrap required, opened a new log
I20260808 00:35:50.733364 23213 ts_tablet_manager.cc:1397] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:35:50.733695 23213 raft_consensus.cc:348] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.733763 23213 raft_consensus.cc:374] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:50.733805 23213 raft_consensus.cc:729] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103be76fe1fc414f9e4c56653a4f2278, State: Initialized, Role: FOLLOWER
I20260808 00:35:50.733855 23213 consensus_queue.cc:260] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.733975 23213 ts_tablet_manager.cc:1428] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:35:50.734148 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.730017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:41261 (local address 127.0.0.1:56262)
0808 00:35:50.730069 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.730081 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.730119 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.730623 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.730624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.730626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:50.730683 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.730685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.732021 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.732023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.733137 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.733139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.733849 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.733850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.733944 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.733968 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.734025 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.734073 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:35:50.734323 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.730017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:45419 (local address 127.0.0.1:45834)
0808 00:35:50.730261 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.730273 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.730326 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.730757 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.730759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.730762 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:50.730807 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.730809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.731763 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.731764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.732942 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.732944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.734053 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.734054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.734139 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.734144 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.734200 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.734244 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":13,"thread_start_us":117,"threads_started":1}
I20260808 00:35:50.734580 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.730259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.31.210.194:45419)
0808 00:35:50.730522 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.730528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.730538 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:35:50.730578 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.730581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.730596 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.730681 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:50.730911 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.731672 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.733077 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.733867 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.733919 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.733975 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.734456 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.734499 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":105,"threads_started":1}
I20260808 00:35:50.734706 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.730221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.31.210.195:41261)
0808 00:35:50.730417 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.730422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.730432 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:35:50.730481 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.730484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.730499 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.730763 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:50.730880 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.731902 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.733230 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.733763 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.733852 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.733911 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.734118 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.734603 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260808 00:35:50.735352 23470 tablet_service.cc:1467] Processing CreateTablet for tablet ff0b9358b14042a5b3d96964031a9ad8 (DEFAULT_TABLE table=test-table [id=b9f035227161421781d65dee6b9a60fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:50.735370 23676 tablet_service.cc:1467] Processing CreateTablet for tablet ff0b9358b14042a5b3d96964031a9ad8 (DEFAULT_TABLE table=test-table [id=b9f035227161421781d65dee6b9a60fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:35:50.735736 23470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff0b9358b14042a5b3d96964031a9ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:50.735752 23676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff0b9358b14042a5b3d96964031a9ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:35:50.738550 23782 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Bootstrap starting.
I20260808 00:35:50.738634 23783 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Bootstrap starting.
I20260808 00:35:50.739207 23783 tablet_bootstrap.cc:654] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:50.739210 23782 tablet_bootstrap.cc:654] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Neither blocks nor log segments found. Creating new log.
I20260808 00:35:50.739708 23782 log.cc:824] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:50.739729 23783 log.cc:824] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Log is configured to *not* fsync() on all Append() calls
I20260808 00:35:50.740423 23783 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: No bootstrap required, opened a new log
I20260808 00:35:50.740435 23782 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: No bootstrap required, opened a new log
I20260808 00:35:50.740527 23783 ts_tablet_manager.cc:1397] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:50.740536 23782 ts_tablet_manager.cc:1397] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:50.742014 23782 raft_consensus.cc:348] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.742177 23782 raft_consensus.cc:374] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:50.742062 23783 raft_consensus.cc:348] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.742201 23782 raft_consensus.cc:729] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f343278d3b4733adc49886f6537903, State: Initialized, Role: FOLLOWER
I20260808 00:35:50.742220 23783 raft_consensus.cc:374] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:35:50.742260 23783 raft_consensus.cc:729] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69746a4ff14245158d14cb5a78399d48, State: Initialized, Role: FOLLOWER
I20260808 00:35:50.742333 23782 consensus_queue.cc:260] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.742378 23783 consensus_queue.cc:260] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.742646 23768 heartbeater.cc:503] Master 127.31.210.254:46623 was elected leader, sending a full tablet report...
I20260808 00:35:50.742689 23783 ts_tablet_manager.cc:1428] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:35:50.742689 23574 heartbeater.cc:503] Master 127.31.210.254:46623 was elected leader, sending a full tablet report...
I20260808 00:35:50.742643 23782 ts_tablet_manager.cc:1428] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260808 00:35:50.800252 23575 tablet.cc:2369] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:35:50.920472 23372 tablet.cc:2369] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:35:50.932082 23791 raft_consensus.cc:482] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:35:50.932226 23791 raft_consensus.cc:504] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.932678 23791 leader_election.cc:288] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087)
I20260808 00:35:50.936790 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.932962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:53653 (local address 127.31.210.195:41261)
0808 00:35:50.933011 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.933013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.933024 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:35:50.933130 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.933131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.933143 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.933223 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:50.933445 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.934225 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.935784 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.936362 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.936446 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.936458 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.936674 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.936714 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:35:50.936800 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.933022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:51231 (local address 127.31.210.193:38087)
0808 00:35:50.933082 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.933084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.933191 (+ 107us) server_negotiation.cc:407] Connection header received
0808 00:35:50.933263 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.933265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.933297 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.933349 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:35:50.933523 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.934250 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.935783 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.936411 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.936510 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.936589 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.936668 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.936714 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:35:50.936813 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.932867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:41261 (local address 127.31.210.194:53653)
0808 00:35:50.932949 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.932962 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.933043 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.933253 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.933255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.933256 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:50.933360 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.933363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.934326 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.934327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.935683 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.935686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.936459 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.936461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.936592 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.936594 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.936656 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.936717 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":79}
I20260808 00:35:50.936820 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.932867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38087 (local address 127.31.210.194:51231)
0808 00:35:50.933114 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.933124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:35:50.933181 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:35:50.933377 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:35:50.933378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:35:50.933379 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:35:50.933443 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.933444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.934326 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.934327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.935683 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:35:50.935686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.936510 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.936511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:35:50.936588 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.936593 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:35:50.936656 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:35:50.936716 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":136,"thread_start_us":102,"threads_started":1}
I20260808 00:35:50.937273 23700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0b9358b14042a5b3d96964031a9ad8" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f343278d3b4733adc49886f6537903" is_pre_election: true
I20260808 00:35:50.937273 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0b9358b14042a5b3d96964031a9ad8" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103be76fe1fc414f9e4c56653a4f2278" is_pre_election: true
I20260808 00:35:50.937448 23301 raft_consensus.cc:2454] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 0.
I20260808 00:35:50.937418 23700 raft_consensus.cc:2454] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 0.
I20260808 00:35:50.937723 23412 leader_election.cc:302] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103be76fe1fc414f9e4c56653a4f2278, 69746a4ff14245158d14cb5a78399d48; no voters:
I20260808 00:35:50.937863 23791 raft_consensus.cc:2781] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:35:50.937922 23791 raft_consensus.cc:482] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:35:50.937952 23791 raft_consensus.cc:3037] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:50.938668 23791 raft_consensus.cc:504] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.938823 23791 leader_election.cc:288] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 election: Requested vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087)
I20260808 00:35:50.938994 23700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0b9358b14042a5b3d96964031a9ad8" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f343278d3b4733adc49886f6537903"
I20260808 00:35:50.939050 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0b9358b14042a5b3d96964031a9ad8" candidate_uuid: "69746a4ff14245158d14cb5a78399d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103be76fe1fc414f9e4c56653a4f2278"
I20260808 00:35:50.939150 23301 raft_consensus.cc:3037] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:50.939100 23700 raft_consensus.cc:3037] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:35:50.939831 23301 raft_consensus.cc:2454] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 1.
I20260808 00:35:50.939934 23700 raft_consensus.cc:2454] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69746a4ff14245158d14cb5a78399d48 in term 1.
I20260808 00:35:50.940002 23412 leader_election.cc:302] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103be76fe1fc414f9e4c56653a4f2278, 69746a4ff14245158d14cb5a78399d48; no voters:
I20260808 00:35:50.940094 23791 raft_consensus.cc:2781] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:35:50.940255 23791 raft_consensus.cc:686] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 1 LEADER]: Becoming Leader. State: Replica: 69746a4ff14245158d14cb5a78399d48, State: Running, Role: LEADER
I20260808 00:35:50.940368 23791 consensus_queue.cc:237] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:35:50.941340 23088 catalog_manager.cc:5495] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69746a4ff14245158d14cb5a78399d48 (127.31.210.194). New cstate: current_term: 1 leader_uuid: "69746a4ff14245158d14cb5a78399d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } health_report { overall_health: UNKNOWN } } }
W20260808 00:35:50.943008 23770 tablet.cc:2369] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:35:50.961054 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.958477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.31.210.194:45419)
0808 00:35:50.958557 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:35:50.958559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:35:50.958570 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:35:50.958625 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:35:50.958625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:35:50.958644 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:35:50.958709 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:35:50.958864 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.959470 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.960181 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:35:50.960380 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:35:50.960464 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:35:50.960682 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:35:50.960791 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:35:50.960861 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:35:50.960892 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 00:35:50.960910 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:35:50.962991 23700 raft_consensus.cc:1261] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Refusing update from remote peer 69746a4ff14245158d14cb5a78399d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:50.962992 23301 raft_consensus.cc:1261] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Refusing update from remote peer 69746a4ff14245158d14cb5a78399d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:35:50.963387 23795 consensus_queue.cc:1035] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:50.963536 23791 consensus_queue.cc:1035] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:35:50.965587 23797 mvcc.cc:205] Tried to move back new op lower bound from 7316067741541363712 to 7316067741452525568. Current Snapshot: MvccSnapshot[applied={T|T < 7316067741541363712}]
I20260808 00:35:50.966040 23799 mvcc.cc:205] Tried to move back new op lower bound from 7316067741541363712 to 7316067741452525568. Current Snapshot: MvccSnapshot[applied={T|T < 7316067741541363712}]
I20260808 00:35:50.966886 23803 mvcc.cc:205] Tried to move back new op lower bound from 7316067741541363712 to 7316067741452525568. Current Snapshot: MvccSnapshot[applied={T|T < 7316067741541363712}]
I20260808 00:35:50.968116 23088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:50.968220 23088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:35:50.969017 23088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Sending DeleteTablet for 3 replicas of tablet ff0b9358b14042a5b3d96964031a9ad8
I20260808 00:35:50.969326 23470 tablet_service.cc:1514] Processing DeleteTablet for tablet ff0b9358b14042a5b3d96964031a9ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:50 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:45834
I20260808 00:35:50.969337 23274 tablet_service.cc:1514] Processing DeleteTablet for tablet ff0b9358b14042a5b3d96964031a9ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:50 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52654
I20260808 00:35:50.969559 23814 tablet_replica.cc:330] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: stopping tablet replica
I20260808 00:35:50.969645 23815 tablet_replica.cc:330] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: stopping tablet replica
I20260808 00:35:50.969655 23814 raft_consensus.cc:2229] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:35:50.969622 23676 tablet_service.cc:1514] Processing DeleteTablet for tablet ff0b9358b14042a5b3d96964031a9ad8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:35:50 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:56262
I20260808 00:35:50.969712 23815 raft_consensus.cc:2229] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:50.969750 23815 raft_consensus.cc:2258] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:50.969761 23814 raft_consensus.cc:2258] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:50.969864 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 23140
I20260808 00:35:50.970157 23814 ts_tablet_manager.cc:1905] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:50.973958 23814 ts_tablet_manager.cc:1918] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:50.974026 23814 log.cc:1196] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/ff0b9358b14042a5b3d96964031a9ad8
I20260808 00:35:50.974273 23814 ts_tablet_manager.cc:1939] T ff0b9358b14042a5b3d96964031a9ad8 P 69746a4ff14245158d14cb5a78399d48: Deleting consensus metadata
I20260808 00:35:50.974658 23067 catalog_manager.cc:4841] TS 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419): tablet ff0b9358b14042a5b3d96964031a9ad8 (table test-table [id=b9f035227161421781d65dee6b9a60fc]) successfully deleted
I20260808 00:35:50.979434 23818 tablet_replica.cc:330] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: stopping tablet replica
I20260808 00:35:50.979640 23818 raft_consensus.cc:2229] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:35:50.979805 23818 raft_consensus.cc:2258] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:35:50.980454 23818 ts_tablet_manager.cc:1905] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:35:50.982524 23818 ts_tablet_manager.cc:1918] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:35:50.982612 23818 log.cc:1196] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/ff0b9358b14042a5b3d96964031a9ad8
I20260808 00:35:50.982906 23818 ts_tablet_manager.cc:1939] T ff0b9358b14042a5b3d96964031a9ad8 P b2f343278d3b4733adc49886f6537903: Deleting consensus metadata
I20260808 00:35:50.983633 23068 catalog_manager.cc:4841] TS b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261): tablet ff0b9358b14042a5b3d96964031a9ad8 (table test-table [id=b9f035227161421781d65dee6b9a60fc]) successfully deleted
W20260808 00:35:50.985647 23069 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:38087 (error 108)
I20260808 00:35:50.986207 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:35:50.986038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:38087 (local address 127.0.0.1:52668)
0808 00:35:50.986121 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 00:35:50.986168 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 00:35:50.986208 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 23376
W20260808 00:35:50.986378 23069 catalog_manager.cc:4584] TS 103be76fe1fc414f9e4c56653a4f2278 (127.31.210.193:38087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff0b9358b14042a5b3d96964031a9ad8: Network error: Client connection negotiation failed: client connection to 127.31.210.193:38087: connect: Connection refused (error 111)
I20260808 00:35:50.996531 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 23581
I20260808 00:35:51.008796 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 23042
I20260808 00:36:11.019704 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46623
--webserver_interface=127.31.210.254
--webserver_port=43139
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:11.112444 23821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:11.112792 23821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:11.112823 23821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:11.114095 23821 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:36:11.114133 23821 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:36:11.114148 23821 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:36:11.114161 23821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:11.114190 23821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:11.114217 23821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:11.114231 23821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:11.114249 23821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:11.115828 23821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=43139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:11.116050 23821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:11.116243 23821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:11.121819 23831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.121870 23834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.122037 23830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:11.122068 23821 server_base.cc:1034] running on GCE node
I20260808 00:36:11.122372 23821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:11.122689 23821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:11.123864 23821 hybrid_clock.cc:648] HybridClock initialized: now 1786149371123850 us; error 53 us; skew 500 ppm
Aug 08 00:36:11 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149371, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:11.132145 23821 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:36:11.132745 23821 webserver.cc:460] Webserver started at http://127.31.210.254:43139/ using document root <none> and password file <none>
I20260808 00:36:11.132962 23821 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:11.133006 23821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:11.134433 23821 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:11.135157 23843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.135347 23821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:11.135442 23821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "835ef069a8134a599e4e670c595f9b7c"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "360a3f0c7af8c219b754e4fad92275d4"
server_key_iv: "cdfca2b0c5d7798389e936ea85388210"
server_key_version: "encryptionkey@0"
I20260808 00:36:11.135788 23821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:11.153173 23821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:11.155509 23821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:11.155656 23821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:11.160413 23821 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46623
I20260808 00:36:11.160555 23928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46623 every 8 connection(s)
I20260808 00:36:11.161019 23821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:11.162637 23929 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:36:11.163384 23929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Bootstrap starting.
I20260808 00:36:11.167162 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 23821
I20260808 00:36:11.167475 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:38087
--local_ip_for_outbound_sockets=127.31.210.193
--tserver_master_addrs=127.31.210.254:46623
--webserver_port=43723
--webserver_interface=127.31.210.193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:36:11.173079 23929 log.cc:824] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:11.175514 23929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:36:11.175829 23929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Bootstrap complete.
I20260808 00:36:11.178344 23929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:36:11.178650 23929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835ef069a8134a599e4e670c595f9b7c, State: Initialized, Role: FOLLOWER
I20260808 00:36:11.178864 23929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:36:11.178928 23929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:11.178987 23929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:11.179037 23929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:36:11.180078 23929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:36:11.180172 23929 leader_election.cc:302] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 835ef069a8134a599e4e670c595f9b7c; no voters:
I20260808 00:36:11.180418 23929 leader_election.cc:288] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 00:36:11.180456 23934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:36:11.180609 23934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [term 3 LEADER]: Becoming Leader. State: Replica: 835ef069a8134a599e4e670c595f9b7c, State: Running, Role: LEADER
I20260808 00:36:11.180755 23929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:11.180718 23934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } }
I20260808 00:36:11.181061 23936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "835ef069a8134a599e4e670c595f9b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } } }
I20260808 00:36:11.181164 23936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:11.181324 23935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 835ef069a8134a599e4e670c595f9b7c. Latest consensus state: current_term: 3 leader_uuid: "835ef069a8134a599e4e670c595f9b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835ef069a8134a599e4e670c595f9b7c" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46623 } } }
I20260808 00:36:11.181380 23935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:11.181588 23941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:11.182655 23941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:11.182773 23941 catalog_manager.cc:1259] Loaded cluster ID: 4d83e14ef3484a428fa458f764aaf770
I20260808 00:36:11.182811 23941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:11.183956 23941 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:11.184165 23941 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Loaded TSK: 0
I20260808 00:36:11.184217 23941 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 835ef069a8134a599e4e670c595f9b7c: Loaded TSK: 1
F20260808 00:36:11.184454 23941 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f754f48f9ad google::LogMessage::Fail() at ??:0
@ 0x7f754f49288a google::LogMessage::SendToLog() at ??:0
@ 0x7f754f48f4de google::LogMessage::Flush() at ??:0
@ 0x7f754f490a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7551bfc104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f7551bf9d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f75573e91cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f75573e948e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f75573e9594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f75573afc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f75573b9ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f755135d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7551352954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f75521b66db start_thread at ??:0
@ 0x7f754df8d61f clone at ??:0
W20260808 00:36:11.268074 23932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:11.268319 23932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:11.268361 23932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:11.269620 23932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:11.269697 23932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:11.269745 23932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:36:11.271354 23932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:38087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=43723
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:11.271657 23932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:11.271905 23932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:11.277513 23988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.277514 23989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.277573 23991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:11.278021 23932 server_base.cc:1034] running on GCE node
I20260808 00:36:11.278281 23932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:11.278596 23932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:11.279798 23932 hybrid_clock.cc:648] HybridClock initialized: now 1786149371279776 us; error 34 us; skew 500 ppm
Aug 08 00:36:11 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149371, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:11.283226 23932 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:36:11.283818 23932 webserver.cc:460] Webserver started at http://127.31.210.193:43723/ using document root <none> and password file <none>
I20260808 00:36:11.284118 23932 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:11.284194 23932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:11.286201 23932 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:11.287068 24007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.287315 23932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:11.287425 23932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "103be76fe1fc414f9e4c56653a4f2278"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "a8b8833b2d51d6aa31a90e9b320e829f"
server_key_iv: "c32e221a89f559e218b84bc89d80d076"
server_key_version: "encryptionkey@0"
I20260808 00:36:11.287926 23932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:11.308735 23932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:11.311548 23932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:11.311739 23932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:11.312058 23932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:11.312739 24017 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:36:11.314448 23932 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:36:11.314522 23932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:36:11.314564 23932 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:36:11.315622 23932 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:36:11.315675 23932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:36:11.315773 24017 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap starting.
I20260808 00:36:11.324282 23932 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:38087
I20260808 00:36:11.324877 23932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:36:11.325510 24172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:38087 every 8 connection(s)
I20260808 00:36:11.327310 24017 log.cc:824] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:11.329694 24017 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:36:11.329998 24017 tablet_bootstrap.cc:492] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Bootstrap complete.
I20260808 00:36:11.331004 24017 ts_tablet_manager.cc:1397] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260808 00:36:11.332068 24017 raft_consensus.cc:348] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:36:11.332327 24017 raft_consensus.cc:729] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103be76fe1fc414f9e4c56653a4f2278, State: Initialized, Role: FOLLOWER
I20260808 00:36:11.332540 24017 consensus_queue.cc:260] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:36:11.332890 24017 ts_tablet_manager.cc:1428] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260808 00:36:11.334772 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 23932
I20260808 00:36:11.335094 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:45419
--local_ip_for_outbound_sockets=127.31.210.194
--tserver_master_addrs=127.31.210.254:46623
--webserver_port=33369
--webserver_interface=127.31.210.194
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:11.418754 24176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:36:11.325741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:48275)
0808 00:36:11.325937 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.325982 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:11.326812 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:11.418527 (+ 91715us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
W20260808 00:36:11.419083 24174 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:46623 (0 consecutive failures): Network error: Failed to ping master at 127.31.210.254:46623: Client connection negotiation failed: client connection to 127.31.210.254:46623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:36:11.419453 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.419313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:47133)
0808 00:36:11.419372 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.419425 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:36:11.419839 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.419736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:51229)
0808 00:36:11.419786 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.419817 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:36:11.419984 24174 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:36:11.424851 24180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:11.425033 24180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:11.425064 24180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:11.426146 24180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:11.426180 24180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:11.426205 24180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:36:11.427641 24180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:45419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=33369
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:11.427840 24180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:11.428021 24180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:36:11.433857 24180 server_base.cc:1034] running on GCE node
W20260808 00:36:11.433836 24188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.433821 24187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.433821 24190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:11.434319 24180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:11.434595 24180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:11.435761 24180 hybrid_clock.cc:648] HybridClock initialized: now 1786149371435750 us; error 44 us; skew 500 ppm
Aug 08 00:36:11 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149371, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:11.438457 24180 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:36:11.438863 24180 webserver.cc:460] Webserver started at http://127.31.210.194:33369/ using document root <none> and password file <none>
I20260808 00:36:11.439071 24180 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:11.439116 24180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:11.440434 24180 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:11.441126 24199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.441345 24180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:36:11.441438 24180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "69746a4ff14245158d14cb5a78399d48"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "ff6f3c4dc52e331d145f15b86212d3c6"
server_key_iv: "363931581c981dbc0e8b99aac9677464"
server_key_version: "encryptionkey@0"
I20260808 00:36:11.441794 24180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:11.465318 24180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:11.467558 24180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:11.467689 24180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:11.467886 24180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:11.468294 24180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:11.468333 24180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.468354 24180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:11.468406 24180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.474785 24180 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:45419
I20260808 00:36:11.474881 24367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:45419 every 8 connection(s)
I20260808 00:36:11.475265 24180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:36:11.476805 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.476191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:44897)
0808 00:36:11.476578 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.476657 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":104,"threads_started":1}
W20260808 00:36:11.477227 24368 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:46623 (0 consecutive failures): Network error: Failed to ping master at 127.31.210.254:46623: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
I20260808 00:36:11.477500 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.477412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:56111)
0808 00:36:11.477456 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.477483 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 00:36:11.477787 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.477712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:60039)
0808 00:36:11.477751 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.477772 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:36:11.477909 24368 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:36:11.483158 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 24180
I20260808 00:36:11.483436 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:41261
--local_ip_for_outbound_sockets=127.31.210.195
--tserver_master_addrs=127.31.210.254:46623
--webserver_port=36249
--webserver_interface=127.31.210.195
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:11.569399 24373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:11.569602 24373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:11.569624 24373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:11.570762 24373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:11.570807 24373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:11.570860 24373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:36:11.572299 24373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:41261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=36249
--tserver_master_addrs=127.31.210.254:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:11.572566 24373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:11.572840 24373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:11.578099 24383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.578142 24385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:11.578150 24381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:11.578545 24373 server_base.cc:1034] running on GCE node
I20260808 00:36:11.578748 24373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:11.578990 24373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:11.580144 24373 hybrid_clock.cc:648] HybridClock initialized: now 1786149371580127 us; error 31 us; skew 500 ppm
Aug 08 00:36:11 dist-test-slave-44d2 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149371, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:11.582701 24373 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:36:11.583187 24373 webserver.cc:460] Webserver started at http://127.31.210.195:36249/ using document root <none> and password file <none>
I20260808 00:36:11.583418 24373 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:11.583487 24373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:11.584998 24373 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:36:11.585664 24393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.585841 24373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.585968 24373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "b2f343278d3b4733adc49886f6537903"
format_stamp: "Formatted at 2026-08-08 00:35:29 on dist-test-slave-44d2"
server_key: "5037ca68b180e383af2c9ad5c918185b"
server_key_iv: "b17086bcdb4672a467586f8f4084fd16"
server_key_version: "encryptionkey@0"
I20260808 00:36:11.586366 24373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:11.593449 24373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:11.595503 24373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:11.595682 24373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:11.595953 24373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:11.596282 24373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:11.596339 24373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.596383 24373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:11.596408 24373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:11.602777 24373 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:41261
I20260808 00:36:11.602828 24560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:41261 every 8 connection(s)
I20260808 00:36:11.603229 24373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:36:11.604453 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.604014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:34471)
0808 00:36:11.604179 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.604291 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
W20260808 00:36:11.604933 24561 heartbeater.cc:644] Failed to heartbeat to 127.31.210.254:46623 (0 consecutive failures): Network error: Failed to ping master at 127.31.210.254:46623: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
I20260808 00:36:11.605224 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.605136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:49621)
0808 00:36:11.605180 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.605207 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:36:11.605572 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:11.605462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:59583)
0808 00:36:11.605529 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:11.605557 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 00:36:11.605679 24561 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:36:11.609792 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 24373
I20260808 00:36:12.420696 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:12.420372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:44637)
0808 00:36:12.420582 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:12.420626 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
I20260808 00:36:12.478513 24568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:12.478272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:36201)
0808 00:36:12.478420 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:12.478452 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260808 00:36:12.606293 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:12.606090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:49977)
0808 00:36:12.606213 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:12.606242 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260808 00:36:12.814730 24570 raft_consensus.cc:482] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:12.814875 24570 raft_consensus.cc:504] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:36:12.815270 24570 leader_election.cc:288] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419)
Aug 08 00:36:12 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149371, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.195@KRBTEST.COM
Aug 08 00:36:12 dist-test-slave-44d2 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149371, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.194@KRBTEST.COM
I20260808 00:36:12.826522 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:12.815431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:41261 (local address 127.31.210.193:38411)
0808 00:36:12.815481 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:12.815495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:12.815566 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:12.816703 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:12.816706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:12.817203 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:12.817489 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:12.817501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.818641 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.818643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:12.819464 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:12.819467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.820113 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.820114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:12.820207 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:12.820945 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:12.820950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:12.822521 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:12.824839 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:12.824847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:12.824850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:12.825159 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:12.825428 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:12.825429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:12.825429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:12.825567 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:12.825964 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:12.825971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:12.826109 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 00:36:12.826222 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 00:36:12.826424 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":68}
I20260808 00:36:12.826534 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:12.815619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:47853 (local address 127.31.210.194:45419)
0808 00:36:12.815797 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:12.815814 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:12.815840 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:36:12.816352 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:12.816355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:12.816400 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:12.816616 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:12.817630 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.818539 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.819577 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.819988 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.820114 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:12.823025 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:12.823029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:12.823030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:12.823042 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:12.824777 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:12.825250 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:12.825253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:12.825254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:12.825351 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:12.825639 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:12.825640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:12.825640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:12.825885 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:12.825980 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:12.826278 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 00:36:12.826412 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260808 00:36:12.826548 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:12.815613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:38411 (local address 127.31.210.195:41261)
0808 00:36:12.815797 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:12.815807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:12.815837 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:36:12.816390 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:12.816393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:12.816432 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:12.816627 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:12.817676 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.818527 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.819616 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.820026 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.820146 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:12.822647 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:12.822651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:12.822652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:12.822682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:12.824760 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:12.825248 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:12.825252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:12.825253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:12.825352 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:12.825640 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:12.825642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:12.825642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:12.825885 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:12.825980 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:12.826278 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 00:36:12.826410 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":62,"threads_started":1}
I20260808 00:36:12.827180 24571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:12.815473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:45419 (local address 127.31.210.193:47853)
0808 00:36:12.815614 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:12.815625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:12.815687 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:12.816751 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:12.816752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:12.817203 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:12.817489 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:12.817496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.818669 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.818670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:12.819465 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:12.819468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:12.820089 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:12.820090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:12.820205 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:12.820878 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:12.820887 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:12.822915 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:12.824882 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:12.824883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:12.824884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:12.825160 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:12.825427 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:12.825428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:12.825428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:12.825564 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:12.825964 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:12.825971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:12.826110 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 00:36:12.826222 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 00:36:12.826515 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":184,"thread_start_us":81,"threads_started":1}
W20260808 00:36:12.827420 24012 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
W20260808 00:36:12.828096 24011 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
I20260808 00:36:12.828158 24011 leader_election.cc:302] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103be76fe1fc414f9e4c56653a4f2278; no voters: 69746a4ff14245158d14cb5a78399d48, b2f343278d3b4733adc49886f6537903
I20260808 00:36:12.828317 24570 raft_consensus.cc:2726] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:36:13.421545 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:13.421299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:48693)
0808 00:36:13.421445 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:13.421490 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260808 00:36:13.479323 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:13.479090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:42313)
0808 00:36:13.479234 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:13.479276 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260808 00:36:13.607040 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:13.606789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:49109)
0808 00:36:13.606920 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:13.606964 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260808 00:36:14.422343 24579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:14.422088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:56995)
0808 00:36:14.422251 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:14.422296 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260808 00:36:14.480130 24580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:14.479870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:43785)
0808 00:36:14.480024 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:14.480073 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260808 00:36:14.517145 24583 raft_consensus.cc:482] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:14.517292 24583 raft_consensus.cc:504] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:36:14.517518 24583 leader_election.cc:288] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419)
W20260808 00:36:14.518124 24012 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
W20260808 00:36:14.518226 24011 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
I20260808 00:36:14.518327 24011 leader_election.cc:302] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103be76fe1fc414f9e4c56653a4f2278; no voters: 69746a4ff14245158d14cb5a78399d48, b2f343278d3b4733adc49886f6537903
I20260808 00:36:14.518411 24583 raft_consensus.cc:2726] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:36:14.607705 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:14.607490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:57423)
0808 00:36:14.607628 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:14.607664 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260808 00:36:15.423286 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:15.422985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:33301)
0808 00:36:15.423187 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:15.423235 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
I20260808 00:36:15.480856 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:15.480642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:58287)
0808 00:36:15.480770 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:15.480807 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260808 00:36:15.608564 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:15.608229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:54395)
0808 00:36:15.608402 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:15.608447 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260808 00:36:16.424098 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:16.423869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:56217)
0808 00:36:16.424012 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:16.424051 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260808 00:36:16.481652 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:16.481379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:55321)
0808 00:36:16.481559 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:16.481604 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260808 00:36:16.609499 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:16.609189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:43757)
0808 00:36:16.609374 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:16.609441 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260808 00:36:17.424943 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:17.424694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:53635)
0808 00:36:17.424849 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:17.424889 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260808 00:36:17.482627 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:17.482232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:32827)
0808 00:36:17.482441 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:17.482522 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
I20260808 00:36:17.597386 24606 raft_consensus.cc:482] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:17.597491 24606 raft_consensus.cc:504] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:36:17.597677 24606 leader_election.cc:288] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419)
W20260808 00:36:17.598119 24012 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
W20260808 00:36:17.598202 24011 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
I20260808 00:36:17.598256 24011 leader_election.cc:302] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103be76fe1fc414f9e4c56653a4f2278; no voters: 69746a4ff14245158d14cb5a78399d48, b2f343278d3b4733adc49886f6537903
I20260808 00:36:17.598330 24606 raft_consensus.cc:2726] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:36:17.610314 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:17.610067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:49583)
0808 00:36:17.610225 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:17.610269 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260808 00:36:18.425817 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:18.425573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:42043)
0808 00:36:18.425721 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:18.425769 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260808 00:36:18.483561 24609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:18.483261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:54357)
0808 00:36:18.483455 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:18.483501 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":67,"threads_started":1}
I20260808 00:36:18.611106 24610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:18.610855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:35831)
0808 00:36:18.610997 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:18.611058 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260808 00:36:19.426571 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:19.426333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:41291)
0808 00:36:19.426478 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:19.426520 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260808 00:36:19.484340 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:19.484110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:32991)
0808 00:36:19.484249 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:19.484292 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260808 00:36:19.611888 24613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:19.611607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:42177)
0808 00:36:19.611790 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:19.611839 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
I20260808 00:36:20.427474 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:20.427170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:39135)
0808 00:36:20.427377 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:20.427427 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
I20260808 00:36:20.485258 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:20.484973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:57607)
0808 00:36:20.485158 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:20.485198 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260808 00:36:20.612752 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:20.612418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:49151)
0808 00:36:20.612651 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:20.612693 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":146,"threads_started":1}
I20260808 00:36:21.428303 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:21.428054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:56011)
0808 00:36:21.428221 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:21.428257 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260808 00:36:21.486222 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:21.485803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:48271)
0808 00:36:21.486122 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:21.486166 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":100,"threads_started":1}
I20260808 00:36:21.613557 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:21.613319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:55621)
0808 00:36:21.613461 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:21.613508 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260808 00:36:21.939605 24620 raft_consensus.cc:482] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:21.939725 24620 raft_consensus.cc:504] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f343278d3b4733adc49886f6537903" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 41261 } } peers { permanent_uuid: "69746a4ff14245158d14cb5a78399d48" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 45419 } } peers { permanent_uuid: "103be76fe1fc414f9e4c56653a4f2278" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 38087 } }
I20260808 00:36:21.939884 24620 leader_election.cc:288] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261), 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419)
W20260808 00:36:21.940450 24011 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69746a4ff14245158d14cb5a78399d48 (127.31.210.194:45419): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
W20260808 00:36:21.940593 24012 leader_election.cc:341] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2f343278d3b4733adc49886f6537903 (127.31.210.195:41261): Not found: Tablet not found: ff0b9358b14042a5b3d96964031a9ad8
I20260808 00:36:21.940636 24012 leader_election.cc:302] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103be76fe1fc414f9e4c56653a4f2278; no voters: 69746a4ff14245158d14cb5a78399d48, b2f343278d3b4733adc49886f6537903
I20260808 00:36:21.940716 24620 raft_consensus.cc:2726] T ff0b9358b14042a5b3d96964031a9ad8 P 103be76fe1fc414f9e4c56653a4f2278 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:36:22.429133 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:22.428859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:47501)
0808 00:36:22.429017 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:22.429069 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260808 00:36:22.486986 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:22.486761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:37287)
0808 00:36:22.486897 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:22.486937 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260808 00:36:22.614413 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:22.614168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:34377)
0808 00:36:22.614317 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:22.614362 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260808 00:36:23.429886 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:23.429662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:35113)
0808 00:36:23.429796 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:23.429838 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260808 00:36:23.487864 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:23.487551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:34255)
0808 00:36:23.487740 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:23.487794 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
I20260808 00:36:23.615336 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:23.615081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:46841)
0808 00:36:23.615238 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:23.615285 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260808 00:36:24.430719 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:24.430444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:52545)
0808 00:36:24.430623 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:24.430671 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260808 00:36:24.488691 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:24.488340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:59479)
0808 00:36:24.488596 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:24.488640 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":72,"threads_started":1}
I20260808 00:36:24.616098 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:24.615847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:38841)
0808 00:36:24.615978 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:24.616019 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260808 00:36:25.431581 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:25.431288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:36379)
0808 00:36:25.431457 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:25.431498 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260808 00:36:25.489710 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:25.489361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:39027)
0808 00:36:25.489612 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:25.489660 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":69,"threads_started":1}
I20260808 00:36:25.616961 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:25.616696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:58273)
0808 00:36:25.616856 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:25.616902 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260808 00:36:26.432518 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:26.432238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.193:42023)
0808 00:36:26.432389 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:26.432427 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260808 00:36:26.490553 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:26.490336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.194:35679)
0808 00:36:26.490465 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:26.490507 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260808 00:36:26.616178 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 23932
I20260808 00:36:26.622951 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:26.622082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46623 (local address 127.31.210.195:60103)
0808 00:36:26.622852 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:26.622890 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.254:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":694,"threads_started":1}
I20260808 00:36:26.648239 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 24180
I20260808 00:36:26.655740 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 24373
2026-08-08T00:36:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58871 ms)
[----------] 30 tests from SecurityITest (248431 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:26 dist-test-slave-44d2 krb5kdc[24644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:26 dist-test-slave-44d2 krb5kdc[24644](info): set up 2 sockets
Aug 08 00:36:26 dist-test-slave-44d2 krb5kdc[24644](info): commencing operation
krb5kdc: starting...
W20260808 00:36:28.015893 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:36:28.149967 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44193
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:44193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:28.240330 24660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:28.240636 24660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:28.240684 24660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:28.242023 24660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:28.242105 24660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:28.242134 24660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:28.242166 24660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:28.242202 24660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:28.243861 24660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:44193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:44193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:28.244186 24660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:28.244467 24660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:28.250618 24669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:28.250607 24671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:28.250633 24660 server_base.cc:1034] running on GCE node
W20260808 00:36:28.250607 24667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:28.251160 24660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:28.251428 24660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:28.252578 24660 hybrid_clock.cc:648] HybridClock initialized: now 1786149388252539 us; error 35 us; skew 500 ppm
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:28.255344 24660 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:36:28.255841 24660 webserver.cc:460] Webserver started at http://127.31.210.254:34115/ using document root <none> and password file <none>
I20260808 00:36:28.256057 24660 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:28.256100 24660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:28.256196 24660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:28.257328 24660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "ea4a029e91e34c89bc5213fb1ddce7f1"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "41c312a816fbb08423ede9102c233a5b"
server_key_iv: "32de9c2e334af5f69e6545ff8d307f21"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.257649 24660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "ea4a029e91e34c89bc5213fb1ddce7f1"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "41c312a816fbb08423ede9102c233a5b"
server_key_iv: "32de9c2e334af5f69e6545ff8d307f21"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.259204 24660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:36:28.260035 24679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.260252 24660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:28.260322 24660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "ea4a029e91e34c89bc5213fb1ddce7f1"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "41c312a816fbb08423ede9102c233a5b"
server_key_iv: "32de9c2e334af5f69e6545ff8d307f21"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.260421 24660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:28.271747 24660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:28.274403 24660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:28.274574 24660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:28.279084 24660 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:44193
I20260808 00:36:28.279120 24755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:44193 every 8 connection(s)
I20260808 00:36:28.279500 24660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:28.280440 24756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:28.282859 24756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1: Bootstrap starting.
I20260808 00:36:28.283435 24756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:28.283680 24756 log.cc:824] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:28.284421 24756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1: No bootstrap required, opened a new log
I20260808 00:36:28.286193 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 24660
I20260808 00:36:28.286310 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:28.286540 32587 external_mini_cluster.cc:1428] Setting key 6be938823cd19aae09c7c33a06091071
I20260808 00:36:28.286399 24756 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44193 } }
I20260808 00:36:28.286584 24756 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:28.286626 24756 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea4a029e91e34c89bc5213fb1ddce7f1, State: Initialized, Role: FOLLOWER
I20260808 00:36:28.286787 24756 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44193 } }
I20260808 00:36:28.286880 24756 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:28.286980 24756 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:28.287050 24756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:28.288245 24756 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44193 } }
I20260808 00:36:28.288380 24756 leader_election.cc:302] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea4a029e91e34c89bc5213fb1ddce7f1; no voters:
I20260808 00:36:28.288581 24756 leader_election.cc:288] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:28.288654 24763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:28.288806 24763 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [term 1 LEADER]: Becoming Leader. State: Replica: ea4a029e91e34c89bc5213fb1ddce7f1, State: Running, Role: LEADER
I20260808 00:36:28.288892 24756 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:28.288933 24763 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44193 } }
I20260808 00:36:28.289417 24764 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44193 } } }
I20260808 00:36:28.289502 24764 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:28.289753 24765 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea4a029e91e34c89bc5213fb1ddce7f1. Latest consensus state: current_term: 1 leader_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4a029e91e34c89bc5213fb1ddce7f1" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 44193 } } }
I20260808 00:36:28.290162 24765 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:28.290305 24778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:28.290923 24778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:28.292706 24778 catalog_manager.cc:1347] Generated new cluster ID: 38731a4577ad4024a895eb87cbb1bfa9
I20260808 00:36:28.292763 24778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:28.311671 24778 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:28.311913 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.287188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46598 (local address 127.31.210.254:44193)
0808 00:36:28.287381 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.287390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.287496 (+ 106us) server_negotiation.cc:407] Connection header received
0808 00:36:28.287668 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.287673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.287718 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.287840 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:28.288761 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.290292 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.291368 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.291890 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.292030 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.307283 (+ 15253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:28.307291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:28.307292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:28.307311 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:28.309716 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.310195 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.310204 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.310204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.310344 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.310589 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.310590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.310591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.310970 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:28.311160 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.311446 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.311606 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260808 00:36:28.312372 24778 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:28.317443 24778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1: Generated new TSK 0
I20260808 00:36:28.317641 24778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:36:28.351869 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:28.440250 24796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:28.440551 24796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:28.440605 24796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:28.441793 24796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:28.441861 24796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:28.441906 24796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:36:28.443411 24796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:28.443684 24796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:28.443926 24796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:28.449301 24806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:28.449311 24805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:28.449301 24811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:28.449692 24796 server_base.cc:1034] running on GCE node
I20260808 00:36:28.449998 24796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:28.450270 24796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:28.451433 24796 hybrid_clock.cc:648] HybridClock initialized: now 1786149388451421 us; error 50 us; skew 500 ppm
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:28.454223 24796 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:36:28.454701 24796 webserver.cc:460] Webserver started at http://127.31.210.193:35821/ using document root <none> and password file <none>
I20260808 00:36:28.454913 24796 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:28.454954 24796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:28.455119 24796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:28.456149 24796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "9b1839f895f147d68b12ee82a118778e"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "ab6ea117ae9aa4c3aa43e5baed4115a3"
server_key_iv: "75a0d5e1f79868d2d76854d3d9211417"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.456460 24796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "9b1839f895f147d68b12ee82a118778e"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "ab6ea117ae9aa4c3aa43e5baed4115a3"
server_key_iv: "75a0d5e1f79868d2d76854d3d9211417"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.457888 24796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:28.458640 24820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.458845 24796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:28.458925 24796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "9b1839f895f147d68b12ee82a118778e"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "ab6ea117ae9aa4c3aa43e5baed4115a3"
server_key_iv: "75a0d5e1f79868d2d76854d3d9211417"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.458978 24796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:28.497470 24796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:28.499544 24796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:28.499668 24796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:28.499890 24796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:28.500176 24796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:28.500208 24796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.500232 24796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:28.500245 24796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.506053 24796 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:33375
I20260808 00:36:28.506166 24980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:33375 every 8 connection(s)
I20260808 00:36:28.506501 24796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:36:28.509372 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 24796
I20260808 00:36:28.509537 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:36:28.509766 32587 external_mini_cluster.cc:1428] Setting key 81448b3d84b08ee98069cf90c76b3f89
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:28.517952 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.507354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:41775 (local address 127.31.210.254:44193)
0808 00:36:28.507419 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.507421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.507659 (+ 238us) server_negotiation.cc:407] Connection header received
0808 00:36:28.508147 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.508148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.508161 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.508219 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:28.509225 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.510096 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.511243 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.511444 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.511536 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.514569 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:28.514570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:28.514571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:28.514578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:28.516130 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.516803 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.516804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.516805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.516864 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.517226 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.517227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.517227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.517340 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:28.517390 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.517809 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.517885 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:36:28.518344 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.507311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44193 (local address 127.31.210.193:41775)
0808 00:36:28.507493 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:28.507538 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:28.508065 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:28.508282 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:28.508285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:28.508915 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:28.509121 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.509126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.510242 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.510243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.511152 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.511154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.511554 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.511555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.511679 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.512267 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:28.512277 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:28.514435 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:28.516233 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:28.516243 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:28.516246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:28.516712 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:28.516947 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:28.516948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:28.516948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:28.517140 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:28.517415 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:28.517423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:28.517634 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 00:36:28.517780 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 00:36:28.518010 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260808 00:36:28.519021 24981 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44193
I20260808 00:36:28.519155 24981 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:28.519354 24981 heartbeater.cc:511] Master 127.31.210.254:44193 requested a full tablet report, sending...
I20260808 00:36:28.519928 24709 ts_manager.cc:194] Registered new tserver with Master: 9b1839f895f147d68b12ee82a118778e (127.31.210.193:33375)
I20260808 00:36:28.520638 24709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:41775
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:36:28.541738 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:28.629640 24992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:28.629873 24992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:28.629904 24992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:28.631161 24992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:28.631211 24992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:28.631240 24992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:36:28.632851 24992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:28.633134 24992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:28.633391 24992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:28.638717 24999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:28.638785 24998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:28.638684 25003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:28.639326 24992 server_base.cc:1034] running on GCE node
I20260808 00:36:28.639564 24992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:28.639834 24992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:28.641114 24992 hybrid_clock.cc:648] HybridClock initialized: now 1786149388641102 us; error 57 us; skew 500 ppm
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:28.643949 24992 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:36:28.644469 24992 webserver.cc:460] Webserver started at http://127.31.210.194:42451/ using document root <none> and password file <none>
I20260808 00:36:28.644766 24992 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:28.644814 24992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:28.644927 24992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:28.646371 24992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "f78ef58685b04dbcba8d76bd68a0e80d"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "357ca6034e2cb94c1b70d14e5b8a83bf"
server_key_iv: "648bc3dfd580bb2400e86546fc30d97b"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.646886 24992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "f78ef58685b04dbcba8d76bd68a0e80d"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "357ca6034e2cb94c1b70d14e5b8a83bf"
server_key_iv: "648bc3dfd580bb2400e86546fc30d97b"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.649019 24992 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:36:28.650125 25017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.650303 24992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:28.650417 24992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "f78ef58685b04dbcba8d76bd68a0e80d"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "357ca6034e2cb94c1b70d14e5b8a83bf"
server_key_iv: "648bc3dfd580bb2400e86546fc30d97b"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.650539 24992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:28.679411 24992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:28.681999 24992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:28.682166 24992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:28.682453 24992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:28.682790 24992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:28.682825 24992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.682847 24992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:28.682895 24992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.689390 24992 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:39239
I20260808 00:36:28.689491 25180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:39239 every 8 connection(s)
I20260808 00:36:28.689875 24992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:28.698721 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 24992
I20260808 00:36:28.698858 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:36:28.699137 32587 external_mini_cluster.cc:1428] Setting key 1f568c2964069366315afb6471a0a995
I20260808 00:36:28.699402 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.690727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:36267 (local address 127.31.210.254:44193)
0808 00:36:28.690804 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.690806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.691042 (+ 236us) server_negotiation.cc:407] Connection header received
0808 00:36:28.691556 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.691557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.691568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.691625 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:28.692658 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.693149 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.694132 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.694323 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.694381 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.696599 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:28.696600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:28.696601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:28.696607 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:28.698049 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.698394 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.698395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.698396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.698475 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.698732 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.698733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.698733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.698888 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:28.698960 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.699239 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.699329 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:36:28.699694 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.690698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44193 (local address 127.31.210.194:36267)
0808 00:36:28.690902 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:28.690947 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:28.691467 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:28.691691 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:28.691695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:28.692277 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:28.692534 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.692542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.693236 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.693237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.694041 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.694043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.694399 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.694400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.694513 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.695072 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:28.695080 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:28.696465 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:28.698117 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:28.698123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:28.698125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:28.698330 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:28.698565 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:28.698566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:28.698566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:28.698664 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:28.698951 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:28.698957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:28.699098 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 00:36:28.699186 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:36:28.699399 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":121,"threads_started":1}
I20260808 00:36:28.700203 25181 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44193
I20260808 00:36:28.700321 25181 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:28.700560 25181 heartbeater.cc:511] Master 127.31.210.254:44193 requested a full tablet report, sending...
I20260808 00:36:28.701004 24709 ts_manager.cc:194] Registered new tserver with Master: f78ef58685b04dbcba8d76bd68a0e80d (127.31.210.194:39239)
I20260808 00:36:28.701512 24709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:36267
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:36:28.728137 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44193
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:28.814736 25186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:28.814947 25186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:28.814972 25186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:28.816213 25186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:28.816260 25186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:28.816287 25186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:36:28.818089 25186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:44193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:28.818334 25186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:28.818576 25186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:36:28.824182 25186 server_base.cc:1034] running on GCE node
W20260808 00:36:28.824127 25193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:28.824120 25192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:28.824146 25196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:28.824676 25186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:28.824946 25186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:28.826105 25186 hybrid_clock.cc:648] HybridClock initialized: now 1786149388826083 us; error 37 us; skew 500 ppm
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:28.829005 25186 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:36:28.829468 25186 webserver.cc:460] Webserver started at http://127.31.210.195:42357/ using document root <none> and password file <none>
I20260808 00:36:28.829746 25186 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:28.829823 25186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:28.829977 25186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:28.831091 25186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "c1e35082dd3a475d990a0fa1f7842d73"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "8e972de5e8951ebbd87271fb84d0467e"
server_key_iv: "e831c11823c1d0184178e84b3089bcc6"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.831485 25186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "c1e35082dd3a475d990a0fa1f7842d73"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "8e972de5e8951ebbd87271fb84d0467e"
server_key_iv: "e831c11823c1d0184178e84b3089bcc6"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.832967 25186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:28.833798 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.834017 25186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:28.834115 25186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "c1e35082dd3a475d990a0fa1f7842d73"
format_stamp: "Formatted at 2026-08-08 00:36:28 on dist-test-slave-44d2"
server_key: "8e972de5e8951ebbd87271fb84d0467e"
server_key_iv: "e831c11823c1d0184178e84b3089bcc6"
server_key_version: "encryptionkey@0"
I20260808 00:36:28.834206 25186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:28.859239 25186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:28.861598 25186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:28.861778 25186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:28.862075 25186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:28.862466 25186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:28.862524 25186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.862578 25186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:28.862607 25186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:28.869374 25186 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:39531
I20260808 00:36:28.869434 25363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:39531 every 8 connection(s)
I20260808 00:36:28.869808 25186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:36:28.874284 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 25186
I20260808 00:36:28.874404 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:36:28.874584 32587 external_mini_cluster.cc:1428] Setting key a4bd07cfc2bf3491f2585bd1aefa6c54
Aug 08 00:36:28 dist-test-slave-44d2 krb5kdc[24644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149388, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:28.880609 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.870708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:46183 (local address 127.31.210.254:44193)
0808 00:36:28.870787 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.870790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.870996 (+ 206us) server_negotiation.cc:407] Connection header received
0808 00:36:28.871502 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.871503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.871515 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.871574 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:28.872616 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.873130 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.874190 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.874496 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.874579 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.877394 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:28.877396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:28.877396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:28.877404 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:28.879143 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.879675 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.879676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.879676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.879732 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.879982 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.879982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.879983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.880106 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:28.880159 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.880421 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.880530 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:36:28.880981 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.870653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:44193 (local address 127.31.210.195:46183)
0808 00:36:28.870863 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:28.870903 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:28.871399 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:28.871632 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:28.871636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:28.872242 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:28.872456 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.872461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.873225 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.873226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.874088 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.874091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.874618 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.874636 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.874803 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.875507 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:28.875516 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:28.877294 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:28.879288 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:28.879296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:28.879298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:28.879573 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:28.879797 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:28.879798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:28.879798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:28.879921 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:28.880167 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:28.880171 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:28.880300 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 00:36:28.880393 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 00:36:28.880607 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260808 00:36:28.881565 25364 heartbeater.cc:347] Connected to a master server at 127.31.210.254:44193
I20260808 00:36:28.881695 25364 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:28.881912 25364 heartbeater.cc:511] Master 127.31.210.254:44193 requested a full tablet report, sending...
I20260808 00:36:28.882323 24709 ts_manager.cc:194] Registered new tserver with Master: c1e35082dd3a475d990a0fa1f7842d73 (127.31.210.195:39531)
I20260808 00:36:28.882689 24709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:46183
I20260808 00:36:28.886543 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:36:28.894291 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.887956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.31.210.254:44193)
0808 00:36:28.888046 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.888048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.888150 (+ 102us) server_negotiation.cc:407] Connection header received
0808 00:36:28.888229 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.888230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.888240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.888314 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:28.889056 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.889585 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.890394 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.890660 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.890732 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.891831 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:28.891833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:28.891833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:28.891841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:28.893057 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.893451 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.893452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.893453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.893516 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:28.893726 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:28.893727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:28.893727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:28.893882 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:28.893987 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.894107 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.894188 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 00:36:28.895856 24709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:36:28.897564 24709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:36:28.904855 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.900577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.31.210.193:33375)
0808 00:36:28.901549 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.901554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.901563 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:28.901599 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.901601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.901616 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.901691 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:28.901884 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.902643 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.903879 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.904378 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.904477 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.904573 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.904722 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.904764 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":428,"threads_started":1}
I20260808 00:36:28.904855 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.900399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:33375 (local address 127.0.0.1:36408)
0808 00:36:28.901386 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:28.901394 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:28.901434 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:28.901766 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:28.901767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:28.901768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:28.901803 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.901804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.902754 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.902755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.903783 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.903786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.904474 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.904476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.904582 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.904598 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:36:28.904702 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:36:28.904761 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":114,"threads_started":1}
I20260808 00:36:28.905742 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.900181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:39531 (local address 127.0.0.1:43652)
0808 00:36:28.901222 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:28.901233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:28.901299 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:28.901525 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:28.901528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:28.901530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:28.901605 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.901607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.903254 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.903254 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.904471 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.904472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.905478 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.905479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.905558 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.905563 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:36:28.905616 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:36:28.905661 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"mutex_wait_us":69,"thread_start_us":99,"threads_started":1}
I20260808 00:36:28.905786 24892 tablet_service.cc:1467] Processing CreateTablet for tablet 321c2761f5c64e6da41cacd4c63d396b (DEFAULT_TABLE table=test-table [id=8b4157ae831648139a56fc7f48e85bff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:28.906008 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.900646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.31.210.194:39239)
0808 00:36:28.901173 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.901177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.901190 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:36:28.901237 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.901240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.901258 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.901358 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:28.901758 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.903093 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.904612 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.905553 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.905609 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.905659 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.905880 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.905928 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":350,"threads_started":1}
I20260808 00:36:28.906036 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.900321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.31.210.195:39531)
0808 00:36:28.901160 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:28.901166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:28.901294 (+ 128us) server_negotiation.cc:407] Connection header received
0808 00:36:28.901423 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:28.901426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:28.901442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:28.901509 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:28.901700 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.903144 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.904594 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.905304 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.905380 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.905440 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:28.905911 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 00:36:28.905951 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":111,"threads_started":1}
I20260808 00:36:28.906231 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:28.900166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:39239 (local address 127.0.0.1:51534)
0808 00:36:28.900279 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:28.900319 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:28.900411 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:28.901581 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:28.901582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:28.901584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:28.901626 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.901628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.903239 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.903240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.904469 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:28.904472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:28.905718 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:28.905719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:28.905779 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:28.905783 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:28.906122 (+ 339us) client_negotiation.cc:240] Negotiation successful
0808 00:36:28.906168 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":34,"thread_start_us":66,"threads_started":1}
I20260808 00:36:28.906294 24892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 321c2761f5c64e6da41cacd4c63d396b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:28.907058 25085 tablet_service.cc:1467] Processing CreateTablet for tablet 321c2761f5c64e6da41cacd4c63d396b (DEFAULT_TABLE table=test-table [id=8b4157ae831648139a56fc7f48e85bff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:28.907460 25085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 321c2761f5c64e6da41cacd4c63d396b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:28.909166 25268 tablet_service.cc:1467] Processing CreateTablet for tablet 321c2761f5c64e6da41cacd4c63d396b (DEFAULT_TABLE table=test-table [id=8b4157ae831648139a56fc7f48e85bff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:28.909515 25268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 321c2761f5c64e6da41cacd4c63d396b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:28.910235 25389 tablet_bootstrap.cc:492] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e: Bootstrap starting.
I20260808 00:36:28.911016 25389 tablet_bootstrap.cc:654] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:28.911075 25390 tablet_bootstrap.cc:492] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Bootstrap starting.
I20260808 00:36:28.911720 25389 log.cc:824] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:28.911890 25390 tablet_bootstrap.cc:654] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:28.912429 25390 log.cc:824] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:28.913146 25389 tablet_bootstrap.cc:492] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e: No bootstrap required, opened a new log
I20260808 00:36:28.913205 25389 ts_tablet_manager.cc:1397] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:36:28.913247 25390 tablet_bootstrap.cc:492] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: No bootstrap required, opened a new log
I20260808 00:36:28.913318 25390 ts_tablet_manager.cc:1397] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:28.914535 25389 raft_consensus.cc:348] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:28.914685 25389 raft_consensus.cc:374] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:28.914709 25389 raft_consensus.cc:729] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b1839f895f147d68b12ee82a118778e, State: Initialized, Role: FOLLOWER
I20260808 00:36:28.914664 25390 raft_consensus.cc:348] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:28.914844 25389 consensus_queue.cc:260] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:28.914947 25390 raft_consensus.cc:374] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:28.915001 25390 raft_consensus.cc:729] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f78ef58685b04dbcba8d76bd68a0e80d, State: Initialized, Role: FOLLOWER
I20260808 00:36:28.915112 25390 consensus_queue.cc:260] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:28.915184 25389 ts_tablet_manager.cc:1428] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:28.915333 24981 heartbeater.cc:503] Master 127.31.210.254:44193 was elected leader, sending a full tablet report...
I20260808 00:36:28.915427 25390 ts_tablet_manager.cc:1428] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:28.915477 25181 heartbeater.cc:503] Master 127.31.210.254:44193 was elected leader, sending a full tablet report...
I20260808 00:36:28.912891 25394 tablet_bootstrap.cc:492] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Bootstrap starting.
I20260808 00:36:28.916363 25394 tablet_bootstrap.cc:654] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:28.916986 25394 log.cc:824] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:28.917732 25394 tablet_bootstrap.cc:492] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: No bootstrap required, opened a new log
I20260808 00:36:28.917821 25394 ts_tablet_manager.cc:1397] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260808 00:36:28.919406 25394 raft_consensus.cc:348] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:28.919603 25394 raft_consensus.cc:374] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:28.919656 25394 raft_consensus.cc:729] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e35082dd3a475d990a0fa1f7842d73, State: Initialized, Role: FOLLOWER
I20260808 00:36:28.919821 25394 consensus_queue.cc:260] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:28.920063 25394 ts_tablet_manager.cc:1428] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:36:28.920136 25364 heartbeater.cc:503] Master 127.31.210.254:44193 was elected leader, sending a full tablet report...
W20260808 00:36:28.940642 25182 tablet.cc:2369] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:29.007319 24983 tablet.cc:2369] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:29.120546 25366 tablet.cc:2369] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:36:29.148382 25397 raft_consensus.cc:482] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:29.148604 25397 raft_consensus.cc:504] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:29.149013 25397 leader_election.cc:288] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1e35082dd3a475d990a0fa1f7842d73 (127.31.210.195:39531), 9b1839f895f147d68b12ee82a118778e (127.31.210.193:33375)
I20260808 00:36:29.152117 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.149357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:42255 (local address 127.31.210.195:39531)
0808 00:36:29.149416 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.149418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.149464 (+ 46us) server_negotiation.cc:407] Connection header received
0808 00:36:29.149497 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.149499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.149509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.149572 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:29.149771 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.150353 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.151327 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.151733 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.151790 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.151798 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.151982 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.152038 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:36:29.152136 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.149268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:39531 (local address 127.31.210.194:42255)
0808 00:36:29.149331 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.149346 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.149397 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.149583 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.149584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.149586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:29.149670 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.149672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.150431 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.150433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.151236 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.151238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.151814 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.151815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.151900 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.151904 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.151958 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.151992 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:36:29.152161 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.149514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:39465 (local address 127.31.210.193:33375)
0808 00:36:29.149604 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.149606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.149639 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:36:29.149705 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.149706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.149717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.149778 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:29.149946 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.150473 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.151404 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.151756 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.151850 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.151859 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.152041 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.152095 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 00:36:29.152148 25400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.149398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:33375 (local address 127.31.210.194:39465)
0808 00:36:29.149582 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.149594 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.149653 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.149791 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.149792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.149794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:29.149858 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.149859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.150562 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.150563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.151314 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.151317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.151878 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.151879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.151939 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.151961 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.152022 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.152059 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":110,"threads_started":1}
I20260808 00:36:29.152621 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321c2761f5c64e6da41cacd4c63d396b" candidate_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e35082dd3a475d990a0fa1f7842d73" is_pre_election: true
I20260808 00:36:29.152796 25296 raft_consensus.cc:2454] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f78ef58685b04dbcba8d76bd68a0e80d in term 0.
I20260808 00:36:29.153071 25018 leader_election.cc:302] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1e35082dd3a475d990a0fa1f7842d73, f78ef58685b04dbcba8d76bd68a0e80d; no voters:
I20260808 00:36:29.153190 24919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321c2761f5c64e6da41cacd4c63d396b" candidate_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b1839f895f147d68b12ee82a118778e" is_pre_election: true
I20260808 00:36:29.153231 25397 raft_consensus.cc:2781] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:36:29.153280 25397 raft_consensus.cc:482] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:36:29.153304 25397 raft_consensus.cc:3037] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:29.153337 24919 raft_consensus.cc:2454] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f78ef58685b04dbcba8d76bd68a0e80d in term 0.
I20260808 00:36:29.154119 25397 raft_consensus.cc:504] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:29.154256 25397 leader_election.cc:288] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [CANDIDATE]: Term 1 election: Requested vote from peers c1e35082dd3a475d990a0fa1f7842d73 (127.31.210.195:39531), 9b1839f895f147d68b12ee82a118778e (127.31.210.193:33375)
I20260808 00:36:29.154419 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321c2761f5c64e6da41cacd4c63d396b" candidate_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1e35082dd3a475d990a0fa1f7842d73"
I20260808 00:36:29.154443 24919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "321c2761f5c64e6da41cacd4c63d396b" candidate_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b1839f895f147d68b12ee82a118778e"
I20260808 00:36:29.154511 24919 raft_consensus.cc:3037] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:29.154523 25296 raft_consensus.cc:3037] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:29.155330 25296 raft_consensus.cc:2454] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f78ef58685b04dbcba8d76bd68a0e80d in term 1.
I20260808 00:36:29.155411 24919 raft_consensus.cc:2454] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f78ef58685b04dbcba8d76bd68a0e80d in term 1.
I20260808 00:36:29.155573 25018 leader_election.cc:302] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b1839f895f147d68b12ee82a118778e, f78ef58685b04dbcba8d76bd68a0e80d; no voters:
I20260808 00:36:29.155681 25397 raft_consensus.cc:2781] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:29.155843 25397 raft_consensus.cc:686] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 1 LEADER]: Becoming Leader. State: Replica: f78ef58685b04dbcba8d76bd68a0e80d, State: Running, Role: LEADER
I20260808 00:36:29.155939 25397 consensus_queue.cc:237] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } }
I20260808 00:36:29.156848 24709 catalog_manager.cc:5495] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d reported cstate change: term changed from 0 to 1, leader changed from <none> to f78ef58685b04dbcba8d76bd68a0e80d (127.31.210.194). New cstate: current_term: 1 leader_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f78ef58685b04dbcba8d76bd68a0e80d" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39239 } health_report { overall_health: HEALTHY } } }
I20260808 00:36:29.191637 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.189170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51546 (local address 127.31.210.194:39239)
0808 00:36:29.189230 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.189234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.189248 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:36:29.189318 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.189319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.189338 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.189402 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:36:29.189498 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.190043 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.190862 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.191100 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.191175 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.191298 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:36:29.191444 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:36:29.191509 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.191551 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.191571 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:36:29.193675 25296 raft_consensus.cc:1261] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 1 FOLLOWER]: Refusing update from remote peer f78ef58685b04dbcba8d76bd68a0e80d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:29.193704 24919 raft_consensus.cc:1261] T 321c2761f5c64e6da41cacd4c63d396b P 9b1839f895f147d68b12ee82a118778e [term 1 FOLLOWER]: Refusing update from remote peer f78ef58685b04dbcba8d76bd68a0e80d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:29.194114 25403 consensus_queue.cc:1035] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b1839f895f147d68b12ee82a118778e" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:29.194247 25397 consensus_queue.cc:1035] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e35082dd3a475d990a0fa1f7842d73" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 39531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:29.197510 25407 mvcc.cc:205] Tried to move back new op lower bound from 7316067898134355968 to 7316067897983315968. Current Snapshot: MvccSnapshot[applied={T|T < 7316067898134355968 or (T in {7316067898134355968})}]
I20260808 00:36:29.197669 25409 mvcc.cc:205] Tried to move back new op lower bound from 7316067898134355968 to 7316067897983315968. Current Snapshot: MvccSnapshot[applied={T|T < 7316067898134355968}]
I20260808 00:36:29.198926 24709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:29.199029 24709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:29.199823 24709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea4a029e91e34c89bc5213fb1ddce7f1: Sending DeleteTablet for 3 replicas of tablet 321c2761f5c64e6da41cacd4c63d396b
I20260808 00:36:29.200155 25085 tablet_service.cc:1514] Processing DeleteTablet for tablet 321c2761f5c64e6da41cacd4c63d396b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:29 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:51534
I20260808 00:36:29.200219 25268 tablet_service.cc:1514] Processing DeleteTablet for tablet 321c2761f5c64e6da41cacd4c63d396b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:29 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:43652
I20260808 00:36:29.200229 24892 tablet_service.cc:1514] Processing DeleteTablet for tablet 321c2761f5c64e6da41cacd4c63d396b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:29 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:36408
I20260808 00:36:29.200536 25422 tablet_replica.cc:330] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: stopping tablet replica
I20260808 00:36:29.200603 25420 tablet_replica.cc:330] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: stopping tablet replica
I20260808 00:36:29.200754 25420 raft_consensus.cc:2229] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:36:29.200749 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 24796
I20260808 00:36:29.200824 25422 raft_consensus.cc:2229] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:29.200941 25420 raft_consensus.cc:2258] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:29.201045 25422 raft_consensus.cc:2258] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:29.201709 25422 ts_tablet_manager.cc:1905] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:29.203665 25422 ts_tablet_manager.cc:1918] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:29.203747 25422 log.cc:1196] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/321c2761f5c64e6da41cacd4c63d396b
I20260808 00:36:29.204099 25422 ts_tablet_manager.cc:1939] T 321c2761f5c64e6da41cacd4c63d396b P c1e35082dd3a475d990a0fa1f7842d73: Deleting consensus metadata
I20260808 00:36:29.204567 24680 catalog_manager.cc:4841] TS c1e35082dd3a475d990a0fa1f7842d73 (127.31.210.195:39531): tablet 321c2761f5c64e6da41cacd4c63d396b (table test-table [id=8b4157ae831648139a56fc7f48e85bff]) successfully deleted
I20260808 00:36:29.211325 25420 ts_tablet_manager.cc:1905] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:29.213207 25420 ts_tablet_manager.cc:1918] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:29.213295 25420 log.cc:1196] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/321c2761f5c64e6da41cacd4c63d396b
I20260808 00:36:29.213562 25420 ts_tablet_manager.cc:1939] T 321c2761f5c64e6da41cacd4c63d396b P f78ef58685b04dbcba8d76bd68a0e80d: Deleting consensus metadata
I20260808 00:36:29.214020 24686 catalog_manager.cc:4841] TS f78ef58685b04dbcba8d76bd68a0e80d (127.31.210.194:39239): tablet 321c2761f5c64e6da41cacd4c63d396b (table test-table [id=8b4157ae831648139a56fc7f48e85bff]) successfully deleted
W20260808 00:36:29.217237 24680 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:33375 (error 108)
I20260808 00:36:29.217578 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 24992
I20260808 00:36:29.219816 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.217696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:33375 (local address 127.0.0.1:36410)
0808 00:36:29.219730 (+ 2034us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.219775 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2003}
W20260808 00:36:29.223069 24680 catalog_manager.cc:4584] TS 9b1839f895f147d68b12ee82a118778e (127.31.210.193:33375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 321c2761f5c64e6da41cacd4c63d396b: Network error: Client connection negotiation failed: client connection to 127.31.210.193:33375: connect: Connection refused (error 111)
I20260808 00:36:29.234797 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 25186
I20260808 00:36:29.247689 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 24660
2026-08-08T00:36:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2580 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T00:36:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:29Z Disabled control of system clock
I20260808 00:36:29.281086 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38173
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:38173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:29.364974 25429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:29.365203 25429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:29.365224 25429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:29.366315 25429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:29.366356 25429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:29.366369 25429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:29.366382 25429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:29.366394 25429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:29.367655 25429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:38173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:38173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:29.367856 25429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:29.368100 25429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:29.373263 25441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:29.373384 25429 server_base.cc:1034] running on GCE node
W20260808 00:36:29.373288 25439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:29.373528 25438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:29.373816 25429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:29.374193 25429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:29.375396 25429 hybrid_clock.cc:648] HybridClock initialized: now 1786149389375380 us; error 44 us; skew 500 ppm
I20260808 00:36:29.376762 25429 webserver.cc:460] Webserver started at http://127.31.210.254:38455/ using document root <none> and password file <none>
I20260808 00:36:29.377008 25429 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:29.377059 25429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:29.377231 25429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:29.378319 25429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "244f0117433f4f7f96f0c77a1edd99b2"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "b27a7541930c13db29c7143d1ac38333"
server_key_iv: "c90a106e61cb5f49b28faa533a50e3cf"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.378700 25429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "244f0117433f4f7f96f0c77a1edd99b2"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "b27a7541930c13db29c7143d1ac38333"
server_key_iv: "c90a106e61cb5f49b28faa533a50e3cf"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.380013 25429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:29.380791 25449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.380960 25429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:36:29.381067 25429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "244f0117433f4f7f96f0c77a1edd99b2"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "b27a7541930c13db29c7143d1ac38333"
server_key_iv: "c90a106e61cb5f49b28faa533a50e3cf"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.381150 25429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:29.394639 25429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:29.395021 25429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:29.395190 25429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:29.399590 25429 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:38173
I20260808 00:36:29.399647 25519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:38173 every 8 connection(s)
I20260808 00:36:29.400033 25429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:29.401144 25520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:29.403543 25520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2: Bootstrap starting.
I20260808 00:36:29.404167 25520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:29.404474 25520 log.cc:824] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:29.405272 25520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2: No bootstrap required, opened a new log
I20260808 00:36:29.405959 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 25429
I20260808 00:36:29.406088 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:29.406304 32587 external_mini_cluster.cc:1428] Setting key 98505f6bb92639f103ed3e1730e9a919
I20260808 00:36:29.407223 25520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244f0117433f4f7f96f0c77a1edd99b2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38173 } }
I20260808 00:36:29.407362 25520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:29.407385 25520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 244f0117433f4f7f96f0c77a1edd99b2, State: Initialized, Role: FOLLOWER
I20260808 00:36:29.407524 25520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244f0117433f4f7f96f0c77a1edd99b2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38173 } }
I20260808 00:36:29.407616 25520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:29.407652 25520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:29.407711 25520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:29.408247 25520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244f0117433f4f7f96f0c77a1edd99b2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38173 } }
I20260808 00:36:29.408339 25520 leader_election.cc:302] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 244f0117433f4f7f96f0c77a1edd99b2; no voters:
I20260808 00:36:29.408564 25520 leader_election.cc:288] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:29.408601 25525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:29.408766 25525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [term 1 LEADER]: Becoming Leader. State: Replica: 244f0117433f4f7f96f0c77a1edd99b2, State: Running, Role: LEADER
I20260808 00:36:29.408839 25520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:29.408907 25525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244f0117433f4f7f96f0c77a1edd99b2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38173 } }
I20260808 00:36:29.409344 25525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 244f0117433f4f7f96f0c77a1edd99b2. Latest consensus state: current_term: 1 leader_uuid: "244f0117433f4f7f96f0c77a1edd99b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244f0117433f4f7f96f0c77a1edd99b2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38173 } } }
I20260808 00:36:29.409433 25527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "244f0117433f4f7f96f0c77a1edd99b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "244f0117433f4f7f96f0c77a1edd99b2" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 38173 } } }
I20260808 00:36:29.409524 25525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:29.409531 25527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:29.409920 25539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:29.410710 25539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:29.412582 25539 catalog_manager.cc:1347] Generated new cluster ID: 6116639eac5f47e280054e8bda7c9f04
I20260808 00:36:29.412663 25539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:29.413290 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.406879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43774 (local address 127.31.210.254:38173)
0808 00:36:29.407118 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.407126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.407150 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:36:29.407247 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.407252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.407314 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.407470 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:29.407775 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.409445 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.410595 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.411267 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.411397 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.412192 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:29.412196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:29.412196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:29.412402 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:29.412445 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:29.412459 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:29.412549 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.412839 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.412925 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260808 00:36:29.439672 25539 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:29.440312 25539 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:29.447570 25539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2: Generated new TSK 0
I20260808 00:36:29.447818 25539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:29.452387 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38173
--builtin_ntp_servers=127.31.210.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:29.539296 25554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:29.539551 25554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:29.539590 25554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:29.540825 25554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:29.540884 25554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:29.540926 25554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:36:29.542457 25554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:29.542729 25554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:29.543093 25554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:29.549454 25566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:29.549479 25563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:29.549502 25564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:29.549892 25554 server_base.cc:1034] running on GCE node
I20260808 00:36:29.550180 25554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:29.550520 25554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:29.551692 25554 hybrid_clock.cc:648] HybridClock initialized: now 1786149389551673 us; error 38 us; skew 500 ppm
I20260808 00:36:29.553159 25554 webserver.cc:460] Webserver started at http://127.31.210.193:42047/ using document root <none> and password file <none>
I20260808 00:36:29.553442 25554 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:29.553493 25554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:29.553635 25554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:29.554783 25554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "17f3104e5f944420b6ee18c5d915620b"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "d2758208e63d7ff9c79c69576210f722"
server_key_iv: "5ab46b79ea803f481c27e255f38df0fa"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.555167 25554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "17f3104e5f944420b6ee18c5d915620b"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "d2758208e63d7ff9c79c69576210f722"
server_key_iv: "5ab46b79ea803f481c27e255f38df0fa"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.556670 25554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:36:29.557538 25575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.557753 25554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:29.557853 25554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "17f3104e5f944420b6ee18c5d915620b"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "d2758208e63d7ff9c79c69576210f722"
server_key_iv: "5ab46b79ea803f481c27e255f38df0fa"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.557911 25554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:29.577427 25554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:29.577725 25554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:29.577834 25554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:29.578037 25554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:29.578367 25554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:29.578398 25554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.578423 25554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:29.578435 25554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.584400 25554 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:34505
I20260808 00:36:29.584455 25756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:34505 every 8 connection(s)
I20260808 00:36:29.584887 25554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:36:29.589037 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 25554
I20260808 00:36:29.589188 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:36:29.589404 32587 external_mini_cluster.cc:1428] Setting key f85fa822cc1755d3edb6437d483add08
I20260808 00:36:29.590919 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.585764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38173 (local address 127.31.210.193:42317)
0808 00:36:29.585981 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.586019 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.586531 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.586749 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.586752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.586761 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:29.587299 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.587304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.587981 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.587982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.588907 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.588911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.589380 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.589381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.589585 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.590225 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:29.590226 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:29.590268 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:29.590298 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:29.590536 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:29.590545 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:29.590548 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.590637 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.590745 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260808 00:36:29.590844 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.585778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:42317 (local address 127.31.210.254:38173)
0808 00:36:29.585845 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.585847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.586118 (+ 271us) server_negotiation.cc:407] Connection header received
0808 00:36:29.586625 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.586626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.586639 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.586693 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:29.587382 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.587891 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.589016 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.589290 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.589355 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.590414 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:29.590415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:29.590415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:29.590423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:29.590443 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:29.590448 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:29.590528 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.590684 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.590724 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:36:29.591063 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38173
--builtin_ntp_servers=127.31.210.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 00:36:29.593205 25758 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38173
I20260808 00:36:29.593335 25758 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:29.593595 25758 heartbeater.cc:511] Master 127.31.210.254:38173 requested a full tablet report, sending...
I20260808 00:36:29.594278 25472 ts_manager.cc:194] Registered new tserver with Master: 17f3104e5f944420b6ee18c5d915620b (127.31.210.193:34505)
I20260808 00:36:29.595034 25472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.193:42317
W20260808 00:36:29.688834 25761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:29.689185 25761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:29.689227 25761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:29.690567 25761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:29.690642 25761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:29.690688 25761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:36:29.692227 25761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:29.692544 25761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:29.692914 25761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:29.698801 25768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:29.698886 25773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:29.699021 25761 server_base.cc:1034] running on GCE node
W20260808 00:36:29.699035 25770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:29.699323 25761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:29.699608 25761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:29.700795 25761 hybrid_clock.cc:648] HybridClock initialized: now 1786149389700778 us; error 26 us; skew 500 ppm
I20260808 00:36:29.702297 25761 webserver.cc:460] Webserver started at http://127.31.210.194:40487/ using document root <none> and password file <none>
I20260808 00:36:29.702581 25761 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:29.702661 25761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:29.702842 25761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:29.703997 25761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "529130c4237b4b089e56bacb859706e4"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "0c844da9ed72dad0b94525b36a94ccae"
server_key_iv: "1922f4f02103796d0ffb35ba5efb4142"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.704412 25761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "529130c4237b4b089e56bacb859706e4"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "0c844da9ed72dad0b94525b36a94ccae"
server_key_iv: "1922f4f02103796d0ffb35ba5efb4142"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.705963 25761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:29.706777 25782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.707062 25761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:29.707196 25761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "529130c4237b4b089e56bacb859706e4"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "0c844da9ed72dad0b94525b36a94ccae"
server_key_iv: "1922f4f02103796d0ffb35ba5efb4142"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.707289 25761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:29.726352 25761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:29.726742 25761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:29.726923 25761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:29.727200 25761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:29.727607 25761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:29.727670 25761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.727717 25761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:29.727751 25761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.734485 25761 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:44601
I20260808 00:36:29.734531 25938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:44601 every 8 connection(s)
I20260808 00:36:29.734933 25761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:36:29.739602 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 25761
I20260808 00:36:29.739737 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:36:29.740113 32587 external_mini_cluster.cc:1428] Setting key 26ae6783c758f0fa936f0f9940bee684
I20260808 00:36:29.741233 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38173
--builtin_ntp_servers=127.31.210.212:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 00:36:29.742128 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.735764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:41101 (local address 127.31.210.254:38173)
0808 00:36:29.735865 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.735868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.736170 (+ 302us) server_negotiation.cc:407] Connection header received
0808 00:36:29.736811 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.736812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.736825 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.736886 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:29.737855 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.738667 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.740220 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.740563 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.740648 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.741851 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:29.741853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:29.741853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:29.741857 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:29.741874 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:29.741879 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:29.741921 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.742042 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.742069 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 00:36:29.742467 25944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.735729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38173 (local address 127.31.210.194:41101)
0808 00:36:29.736018 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.736057 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.736700 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.736965 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.736969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.736978 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:29.737731 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.737738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.738794 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.738796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.740062 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.740066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.740732 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.740735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.740933 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.741670 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:29.741671 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:29.741712 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:29.741742 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:29.741944 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:29.741952 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:29.741955 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.742040 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.742178 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":126,"threads_started":1}
I20260808 00:36:29.743225 25942 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38173
I20260808 00:36:29.743337 25942 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:29.743522 25942 heartbeater.cc:511] Master 127.31.210.254:38173 requested a full tablet report, sending...
I20260808 00:36:29.743963 25472 ts_manager.cc:194] Registered new tserver with Master: 529130c4237b4b089e56bacb859706e4 (127.31.210.194:44601)
I20260808 00:36:29.744328 25472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.194:41101
W20260808 00:36:29.839041 25945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:29.839286 25945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:29.839311 25945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:29.840646 25945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:29.840696 25945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:29.840726 25945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:36:29.842204 25945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:29.842414 25945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:29.842700 25945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:29.848428 25950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:29.848433 25956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:29.848601 25951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:29.848738 25945 server_base.cc:1034] running on GCE node
I20260808 00:36:29.849136 25945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:29.849500 25945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:29.850672 25945 hybrid_clock.cc:648] HybridClock initialized: now 1786149389850650 us; error 39 us; skew 500 ppm
I20260808 00:36:29.852169 25945 webserver.cc:460] Webserver started at http://127.31.210.195:32955/ using document root <none> and password file <none>
I20260808 00:36:29.852469 25945 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:29.852623 25945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:29.852804 25945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:29.853978 25945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "40f4e45a8c604a66a1cedc9463e620a5"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "0583ea13a7373cd2402b65bb6f6632ea"
server_key_iv: "bd3fe4225772a472ed98ed5a1a5a38f7"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.854387 25945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "40f4e45a8c604a66a1cedc9463e620a5"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "0583ea13a7373cd2402b65bb6f6632ea"
server_key_iv: "bd3fe4225772a472ed98ed5a1a5a38f7"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.855861 25945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:36:29.856700 25968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.856890 25945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:36:29.857007 25945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "40f4e45a8c604a66a1cedc9463e620a5"
format_stamp: "Formatted at 2026-08-08 00:36:29 on dist-test-slave-44d2"
server_key: "0583ea13a7373cd2402b65bb6f6632ea"
server_key_iv: "bd3fe4225772a472ed98ed5a1a5a38f7"
server_key_version: "encryptionkey@0"
I20260808 00:36:29.857091 25945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:29.864261 25945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:29.864619 25945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:29.864787 25945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:29.865041 25945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:29.865398 25945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:29.865460 25945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.865505 25945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:29.865536 25945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:29.872428 25945 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:38233
I20260808 00:36:29.872467 26124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:38233 every 8 connection(s)
I20260808 00:36:29.872938 25945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:36:29.878670 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 25945
I20260808 00:36:29.878824 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:36:29.879103 32587 external_mini_cluster.cc:1428] Setting key 2fa9c0398d1d16f86a014f91454c18c0
I20260808 00:36:29.879266 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.873817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:39405 (local address 127.31.210.254:38173)
0808 00:36:29.873890 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.873892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.874283 (+ 391us) server_negotiation.cc:407] Connection header received
0808 00:36:29.874891 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.874892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.874908 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.874956 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:29.875781 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.876372 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.877541 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.877766 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.877865 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.878863 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:29.878865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:29.878865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:29.878871 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:29.878888 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:29.878892 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:29.879006 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.879140 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.879176 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:36:29.879623 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.873809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:38173 (local address 127.31.210.195:39405)
0808 00:36:29.874121 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.874178 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.874763 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.875087 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.875091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.875100 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:29.875673 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.875678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.876521 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.876523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.877422 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.877425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.877877 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.877879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.878032 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.878682 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:29.878683 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:29.878715 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:29.878742 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:29.879041 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:29.879052 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:29.879055 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.879143 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.879260 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":169,"threads_started":1}
I20260808 00:36:29.880203 26125 heartbeater.cc:347] Connected to a master server at 127.31.210.254:38173
I20260808 00:36:29.880331 26125 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:29.880586 26125 heartbeater.cc:511] Master 127.31.210.254:38173 requested a full tablet report, sending...
I20260808 00:36:29.881064 25470 ts_manager.cc:194] Registered new tserver with Master: 40f4e45a8c604a66a1cedc9463e620a5 (127.31.210.195:38233)
I20260808 00:36:29.881457 25470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.195:39405
I20260808 00:36:29.891551 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:36:29.896332 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.893260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43788 (local address 127.31.210.254:38173)
0808 00:36:29.893318 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.893321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.893373 (+ 52us) server_negotiation.cc:407] Connection header received
0808 00:36:29.893444 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.893444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.893458 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.893552 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:29.893765 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.894277 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.895066 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.895257 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.895331 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.896068 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:29.896069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:29.896070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:29.896074 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:29.896095 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:29.896100 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:29.896161 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.896217 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.896244 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:36:29.898030 25470 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:36:29.900061 25470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:36:29.907549 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.902449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:34505 (local address 127.0.0.1:35630)
0808 00:36:29.902767 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.902803 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.902880 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.903210 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.903213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.903216 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:29.903298 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.903300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.904805 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.904806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.906021 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.906025 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.907131 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.907132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.907226 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.907244 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.907322 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.907388 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":30,"thread_start_us":83,"threads_started":1}
I20260808 00:36:29.907585 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.902552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.31.210.193:34505)
0808 00:36:29.902852 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.902857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.902881 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:36:29.902968 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.902971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.902986 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.903108 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:29.903401 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.904699 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.906127 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.907007 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.907098 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.907197 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.907352 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.907419 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":87,"threads_started":1}
I20260808 00:36:29.907768 26140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.902548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.31.210.194:44601)
0808 00:36:29.902746 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.902752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.902968 (+ 216us) server_negotiation.cc:407] Connection header received
0808 00:36:29.903062 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.903066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.903083 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.903162 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:29.903401 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.904178 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.905446 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.906185 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.906270 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.906365 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.907607 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.907672 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260808 00:36:29.907562 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.902435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:44601 (local address 127.0.0.1:46538)
0808 00:36:29.902751 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.902796 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.902877 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.903254 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.903255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.903257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:29.903316 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.903318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.904265 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.904266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.905339 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.905342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.907241 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.907242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.907344 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.907348 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.907411 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.907454 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":74,"threads_started":1}
I20260808 00:36:29.907919 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.902878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.31.210.195:38233)
0808 00:36:29.903282 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:29.903286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:29.903297 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:36:29.903333 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:29.903335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:29.903350 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:29.903420 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:29.903620 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.904828 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.906274 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.907043 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.907118 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.907198 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:29.907746 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 00:36:29.907801 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":82,"threads_started":1}
I20260808 00:36:29.908373 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:29.902668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:38233 (local address 127.0.0.1:54748)
0808 00:36:29.903081 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:29.903091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:29.903139 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:29.903484 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:29.903485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:29.903487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:29.903529 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.903530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.904911 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.904912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.906188 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:29.906191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:29.907129 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:29.907131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:29.907226 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:29.907244 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:36:29.907323 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:36:29.907388 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":177,"thread_start_us":81,"threads_started":1}
I20260808 00:36:29.908725 25651 tablet_service.cc:1467] Processing CreateTablet for tablet 229d069ad9794ed3a6980d5bcb704ddc (DEFAULT_TABLE table=test-table [id=f71db813ff334425a0fafb72257c3a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:29.908730 25847 tablet_service.cc:1467] Processing CreateTablet for tablet 229d069ad9794ed3a6980d5bcb704ddc (DEFAULT_TABLE table=test-table [id=f71db813ff334425a0fafb72257c3a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:29.909142 25651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 229d069ad9794ed3a6980d5bcb704ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:29.909179 25847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 229d069ad9794ed3a6980d5bcb704ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:29.909243 26034 tablet_service.cc:1467] Processing CreateTablet for tablet 229d069ad9794ed3a6980d5bcb704ddc (DEFAULT_TABLE table=test-table [id=f71db813ff334425a0fafb72257c3a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:29.909644 26034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 229d069ad9794ed3a6980d5bcb704ddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:29.912178 26143 tablet_bootstrap.cc:492] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b: Bootstrap starting.
I20260808 00:36:29.912976 26143 tablet_bootstrap.cc:654] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:29.913336 26143 log.cc:824] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:29.913358 26145 tablet_bootstrap.cc:492] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Bootstrap starting.
I20260808 00:36:29.913671 26146 tablet_bootstrap.cc:492] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Bootstrap starting.
I20260808 00:36:29.913982 26145 tablet_bootstrap.cc:654] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:29.914273 26145 log.cc:824] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:29.914324 26146 tablet_bootstrap.cc:654] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:29.914707 26146 log.cc:824] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:29.915225 26145 tablet_bootstrap.cc:492] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: No bootstrap required, opened a new log
I20260808 00:36:29.915319 26145 ts_tablet_manager.cc:1397] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:36:29.915534 26146 tablet_bootstrap.cc:492] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: No bootstrap required, opened a new log
I20260808 00:36:29.915544 26143 tablet_bootstrap.cc:492] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b: No bootstrap required, opened a new log
I20260808 00:36:29.915617 26143 ts_tablet_manager.cc:1397] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 00:36:29.915617 26146 ts_tablet_manager.cc:1397] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:36:29.917376 26145 raft_consensus.cc:348] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:29.917541 26145 raft_consensus.cc:374] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:29.917604 26145 raft_consensus.cc:729] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529130c4237b4b089e56bacb859706e4, State: Initialized, Role: FOLLOWER
I20260808 00:36:29.917734 26145 consensus_queue.cc:260] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:29.918054 25942 heartbeater.cc:503] Master 127.31.210.254:38173 was elected leader, sending a full tablet report...
I20260808 00:36:29.918205 26145 ts_tablet_manager.cc:1428] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:36:29.918607 26146 raft_consensus.cc:348] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:29.918792 26146 raft_consensus.cc:374] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:29.918707 26143 raft_consensus.cc:348] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:29.918857 26143 raft_consensus.cc:374] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:29.918910 26143 raft_consensus.cc:729] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f3104e5f944420b6ee18c5d915620b, State: Initialized, Role: FOLLOWER
I20260808 00:36:29.918946 26146 raft_consensus.cc:729] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40f4e45a8c604a66a1cedc9463e620a5, State: Initialized, Role: FOLLOWER
I20260808 00:36:29.919068 26143 consensus_queue.cc:260] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:29.919076 26146 consensus_queue.cc:260] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:29.919360 26146 ts_tablet_manager.cc:1428] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:36:29.919360 26143 ts_tablet_manager.cc:1428] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260808 00:36:29.919504 25758 heartbeater.cc:503] Master 127.31.210.254:38173 was elected leader, sending a full tablet report...
I20260808 00:36:29.919834 26125 heartbeater.cc:503] Master 127.31.210.254:38173 was elected leader, sending a full tablet report...
W20260808 00:36:29.985601 25943 tablet.cc:2369] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:30.085829 25759 tablet.cc:2369] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:30.123610 26126 tablet.cc:2369] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:36:30.158300 26152 raft_consensus.cc:482] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:30.158440 26152 raft_consensus.cc:504] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:30.158818 26152 leader_election.cc:288] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 529130c4237b4b089e56bacb859706e4 (127.31.210.194:44601), 40f4e45a8c604a66a1cedc9463e620a5 (127.31.210.195:38233)
I20260808 00:36:30.162104 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:30.159175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:42367 (local address 127.31.210.195:38233)
0808 00:36:30.159224 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:30.159226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:30.159261 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:36:30.159376 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:30.159377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:30.159392 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:30.159456 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:30.159698 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.160156 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.161345 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.161723 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.161817 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:30.161827 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:30.161980 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 00:36:30.162022 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:36:30.162118 26160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:30.159037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:44601 (local address 127.31.210.193:56971)
0808 00:36:30.159208 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:30.159218 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:30.159298 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:30.159454 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:30.159454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:30.159456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:30.159533 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:30.159536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.160307 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.160308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:30.161238 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:30.161240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.161840 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.161841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:30.161909 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:30.161912 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:30.161987 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:36:30.162024 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
I20260808 00:36:30.162122 26140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:30.159120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:56971 (local address 127.31.210.194:44601)
0808 00:36:30.159182 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:30.159184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:30.159282 (+ 98us) server_negotiation.cc:407] Connection header received
0808 00:36:30.159356 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:30.159357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:30.159372 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:30.159448 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:30.159668 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.160214 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.161342 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.161723 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.161817 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:30.161826 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:30.162010 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 00:36:30.162055 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:36:30.162139 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:30.159035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:38233 (local address 127.31.210.193:42367)
0808 00:36:30.159197 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:30.159209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:30.159279 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:30.159481 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:30.159482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:30.159484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:30.159622 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:30.159624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.160260 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.160262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:30.161232 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:30.161235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.161816 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.161817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:30.161889 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:30.161893 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:30.161945 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:36:30.162000 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260808 00:36:30.162479 25875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229d069ad9794ed3a6980d5bcb704ddc" candidate_uuid: "17f3104e5f944420b6ee18c5d915620b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529130c4237b4b089e56bacb859706e4" is_pre_election: true
I20260808 00:36:30.162688 25875 raft_consensus.cc:2454] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f3104e5f944420b6ee18c5d915620b in term 0.
I20260808 00:36:30.162683 26062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229d069ad9794ed3a6980d5bcb704ddc" candidate_uuid: "17f3104e5f944420b6ee18c5d915620b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40f4e45a8c604a66a1cedc9463e620a5" is_pre_election: true
I20260808 00:36:30.162847 26062 raft_consensus.cc:2454] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f3104e5f944420b6ee18c5d915620b in term 0.
I20260808 00:36:30.162953 25576 leader_election.cc:302] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f3104e5f944420b6ee18c5d915620b, 529130c4237b4b089e56bacb859706e4; no voters:
I20260808 00:36:30.163092 26152 raft_consensus.cc:2781] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:36:30.163146 26152 raft_consensus.cc:482] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:36:30.163168 26152 raft_consensus.cc:3037] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:30.163880 26152 raft_consensus.cc:504] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:30.164031 26152 leader_election.cc:288] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [CANDIDATE]: Term 1 election: Requested vote from peers 529130c4237b4b089e56bacb859706e4 (127.31.210.194:44601), 40f4e45a8c604a66a1cedc9463e620a5 (127.31.210.195:38233)
I20260808 00:36:30.164170 25875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229d069ad9794ed3a6980d5bcb704ddc" candidate_uuid: "17f3104e5f944420b6ee18c5d915620b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529130c4237b4b089e56bacb859706e4"
I20260808 00:36:30.164175 26062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229d069ad9794ed3a6980d5bcb704ddc" candidate_uuid: "17f3104e5f944420b6ee18c5d915620b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40f4e45a8c604a66a1cedc9463e620a5"
I20260808 00:36:30.164245 25875 raft_consensus.cc:3037] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:30.164268 26062 raft_consensus.cc:3037] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:30.165048 25875 raft_consensus.cc:2454] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f3104e5f944420b6ee18c5d915620b in term 1.
I20260808 00:36:30.165149 26062 raft_consensus.cc:2454] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f3104e5f944420b6ee18c5d915620b in term 1.
I20260808 00:36:30.165223 25576 leader_election.cc:302] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f3104e5f944420b6ee18c5d915620b, 529130c4237b4b089e56bacb859706e4; no voters:
I20260808 00:36:30.165311 26152 raft_consensus.cc:2781] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:30.165459 26152 raft_consensus.cc:686] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [term 1 LEADER]: Becoming Leader. State: Replica: 17f3104e5f944420b6ee18c5d915620b, State: Running, Role: LEADER
I20260808 00:36:30.165573 26152 consensus_queue.cc:237] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } }
I20260808 00:36:30.166421 25470 catalog_manager.cc:5495] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f3104e5f944420b6ee18c5d915620b (127.31.210.193). New cstate: current_term: 1 leader_uuid: "17f3104e5f944420b6ee18c5d915620b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f3104e5f944420b6ee18c5d915620b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 } health_report { overall_health: UNKNOWN } } }
I20260808 00:36:30.192574 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:30.189947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.31.210.193:34505)
0808 00:36:30.190018 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:30.190020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:30.190029 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:30.190084 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:30.190085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:30.190106 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:30.190181 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:36:30.190340 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.190819 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.191672 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:30.191900 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:30.192004 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:30.192194 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:36:30.192311 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:36:30.192410 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:30.192429 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 00:36:30.192450 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:36:30.194514 26062 raft_consensus.cc:1261] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 1 FOLLOWER]: Refusing update from remote peer 17f3104e5f944420b6ee18c5d915620b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:30.194541 25875 raft_consensus.cc:1261] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 1 FOLLOWER]: Refusing update from remote peer 17f3104e5f944420b6ee18c5d915620b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:30.194939 26161 consensus_queue.cc:1035] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [LEADER]: Connected to new peer: Peer: permanent_uuid: "529130c4237b4b089e56bacb859706e4" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 44601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:30.195106 26152 consensus_queue.cc:1035] T 229d069ad9794ed3a6980d5bcb704ddc P 17f3104e5f944420b6ee18c5d915620b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40f4e45a8c604a66a1cedc9463e620a5" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:30.197698 26170 mvcc.cc:205] Tried to move back new op lower bound from 7316067902233800704 to 7316067902118924288. Current Snapshot: MvccSnapshot[applied={T|T < 7316067902233800704}]
I20260808 00:36:30.199851 25472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:30.199950 25472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:30.200793 25472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 244f0117433f4f7f96f0c77a1edd99b2: Sending DeleteTablet for 3 replicas of tablet 229d069ad9794ed3a6980d5bcb704ddc
I20260808 00:36:30.201200 25847 tablet_service.cc:1514] Processing DeleteTablet for tablet 229d069ad9794ed3a6980d5bcb704ddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:30 UTC) from {username='slave'} at 127.0.0.1:46538
I20260808 00:36:30.201203 25651 tablet_service.cc:1514] Processing DeleteTablet for tablet 229d069ad9794ed3a6980d5bcb704ddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:30 UTC) from {username='slave'} at 127.0.0.1:35630
I20260808 00:36:30.201427 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 25554
I20260808 00:36:30.201567 26034 tablet_service.cc:1514] Processing DeleteTablet for tablet 229d069ad9794ed3a6980d5bcb704ddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:30 UTC) from {username='slave'} at 127.0.0.1:54748
I20260808 00:36:30.203881 26185 tablet_replica.cc:330] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: stopping tablet replica
I20260808 00:36:30.201465 26184 tablet_replica.cc:330] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: stopping tablet replica
I20260808 00:36:30.204017 26185 raft_consensus.cc:2229] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:30.204041 26184 raft_consensus.cc:2229] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:30.204130 26185 raft_consensus.cc:2258] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:30.204201 26184 raft_consensus.cc:2258] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:30.204592 26185 ts_tablet_manager.cc:1905] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:30.204695 26184 ts_tablet_manager.cc:1905] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:30.206521 26185 ts_tablet_manager.cc:1918] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:30.206687 26185 log.cc:1196] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/229d069ad9794ed3a6980d5bcb704ddc
I20260808 00:36:30.206801 26184 ts_tablet_manager.cc:1918] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:30.206943 26184 log.cc:1196] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/229d069ad9794ed3a6980d5bcb704ddc
I20260808 00:36:30.207108 26185 ts_tablet_manager.cc:1939] T 229d069ad9794ed3a6980d5bcb704ddc P 40f4e45a8c604a66a1cedc9463e620a5: Deleting consensus metadata
I20260808 00:36:30.207232 26184 ts_tablet_manager.cc:1939] T 229d069ad9794ed3a6980d5bcb704ddc P 529130c4237b4b089e56bacb859706e4: Deleting consensus metadata
I20260808 00:36:30.207597 25451 catalog_manager.cc:4841] TS 40f4e45a8c604a66a1cedc9463e620a5 (127.31.210.195:38233): tablet 229d069ad9794ed3a6980d5bcb704ddc (table test-table [id=f71db813ff334425a0fafb72257c3a7c]) successfully deleted
I20260808 00:36:30.207667 25450 catalog_manager.cc:4841] TS 529130c4237b4b089e56bacb859706e4 (127.31.210.194:44601): tablet 229d069ad9794ed3a6980d5bcb704ddc (table test-table [id=f71db813ff334425a0fafb72257c3a7c]) successfully deleted
W20260808 00:36:30.213745 25451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:34505 (error 108)
I20260808 00:36:30.214008 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 25761
I20260808 00:36:30.215505 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:30.214147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:34505 (local address 127.0.0.1:35170)
0808 00:36:30.215405 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:30.215457 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:34505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1230}
W20260808 00:36:30.215864 25451 catalog_manager.cc:4584] TS 17f3104e5f944420b6ee18c5d915620b (127.31.210.193:34505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 229d069ad9794ed3a6980d5bcb704ddc: Network error: Client connection negotiation failed: client connection to 127.31.210.193:34505: connect: Connection refused (error 111)
I20260808 00:36:30.222793 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 25945
I20260808 00:36:30.230136 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 25429
2026-08-08T00:36:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (984 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:30 dist-test-slave-44d2 krb5kdc[26191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:30 dist-test-slave-44d2 krb5kdc[26191](info): set up 2 sockets
Aug 08 00:36:30 dist-test-slave-44d2 krb5kdc[26191](info): commencing operation
krb5kdc: starting...
W20260808 00:36:31.521134 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:31 dist-test-slave-44d2 krb5kdc[26191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:36:31.660856 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46415
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:31.748814 26213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:31.749081 26213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:31.749122 26213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:31.750366 26213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:31.750442 26213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:31.750464 26213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:31.750486 26213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:31.750517 26213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:31.752106 26213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:31.752404 26213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:31.752687 26213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:36:31.758133 26213 server_base.cc:1034] running on GCE node
W20260808 00:36:31.758085 26218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:31.758057 26222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:31.758119 26219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:31.758708 26213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:31.759042 26213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:31.760242 26213 hybrid_clock.cc:648] HybridClock initialized: now 1786149391760225 us; error 28 us; skew 500 ppm
Aug 08 00:36:31 dist-test-slave-44d2 krb5kdc[26191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149391, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:31.768064 26213 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:36:31.768648 26213 webserver.cc:460] Webserver started at http://127.31.210.254:43007/ using document root <none> and password file <none>
I20260808 00:36:31.768857 26213 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:31.768900 26213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:31.768997 26213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:31.770067 26213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "796944ac1d91475fbfc31e3b62ac5540"
format_stamp: "Formatted at 2026-08-08 00:36:31 on dist-test-slave-44d2"
server_key: "27c3a744ccad06be4ef150be425575b1"
server_key_iv: "2bb1c505081b7c32ef8da96c98063c9c"
server_key_version: "encryptionkey@0"
I20260808 00:36:31.770383 26213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "796944ac1d91475fbfc31e3b62ac5540"
format_stamp: "Formatted at 2026-08-08 00:36:31 on dist-test-slave-44d2"
server_key: "27c3a744ccad06be4ef150be425575b1"
server_key_iv: "2bb1c505081b7c32ef8da96c98063c9c"
server_key_version: "encryptionkey@0"
I20260808 00:36:31.771710 26213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:31.772421 26232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:31.772639 26213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:31.772754 26213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "796944ac1d91475fbfc31e3b62ac5540"
format_stamp: "Formatted at 2026-08-08 00:36:31 on dist-test-slave-44d2"
server_key: "27c3a744ccad06be4ef150be425575b1"
server_key_iv: "2bb1c505081b7c32ef8da96c98063c9c"
server_key_version: "encryptionkey@0"
I20260808 00:36:31.772832 26213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:31.787228 26213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:31.789487 26213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:31.789690 26213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:31.794129 26213 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46415
I20260808 00:36:31.794195 26305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46415 every 8 connection(s)
I20260808 00:36:31.794637 26213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:31.795586 26306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:31.797488 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 26213
I20260808 00:36:31.797621 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:31.797847 32587 external_mini_cluster.cc:1428] Setting key 0de98d6ee6872c9464db7a94687f5f9b
I20260808 00:36:31.798506 26306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540: Bootstrap starting.
I20260808 00:36:31.799218 26306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:31.799576 26306 log.cc:824] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:31.800338 26306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540: No bootstrap required, opened a new log
I20260808 00:36:31.802186 26306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796944ac1d91475fbfc31e3b62ac5540" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46415 } }
I20260808 00:36:31.802349 26306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:31.802415 26306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796944ac1d91475fbfc31e3b62ac5540, State: Initialized, Role: FOLLOWER
I20260808 00:36:31.802557 26306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796944ac1d91475fbfc31e3b62ac5540" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46415 } }
I20260808 00:36:31.802662 26306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:31.802707 26306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:31.802750 26306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:31.803296 26306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796944ac1d91475fbfc31e3b62ac5540" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46415 } }
I20260808 00:36:31.803390 26306 leader_election.cc:302] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 796944ac1d91475fbfc31e3b62ac5540; no voters:
I20260808 00:36:31.803577 26306 leader_election.cc:288] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:31.803601 26316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:31.803721 26316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [term 1 LEADER]: Becoming Leader. State: Replica: 796944ac1d91475fbfc31e3b62ac5540, State: Running, Role: LEADER
I20260808 00:36:31.803862 26306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:31.803869 26316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796944ac1d91475fbfc31e3b62ac5540" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46415 } }
I20260808 00:36:31.804335 26317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 796944ac1d91475fbfc31e3b62ac5540. Latest consensus state: current_term: 1 leader_uuid: "796944ac1d91475fbfc31e3b62ac5540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796944ac1d91475fbfc31e3b62ac5540" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46415 } } }
I20260808 00:36:31.804431 26317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [sys.catalog]: This master's current role is: LEADER
Aug 08 00:36:31 dist-test-slave-44d2 krb5kdc[26191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:31.804715 26318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "796944ac1d91475fbfc31e3b62ac5540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "796944ac1d91475fbfc31e3b62ac5540" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46415 } } }
I20260808 00:36:31.805064 26318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:31.805263 26327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:31.805887 26327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:31.807418 26327 catalog_manager.cc:1347] Generated new cluster ID: 447df3dd80394332a617128ad91d2260
I20260808 00:36:31.807471 26327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:31.809599 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:31.798454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.31.210.254:46415)
0808 00:36:31.798656 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:31.798665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:31.798692 (+ 27us) server_negotiation.cc:407] Connection header received
0808 00:36:31.799147 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:31.799154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:31.799202 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:31.799379 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:31.800134 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:31.801114 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:31.801903 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:31.802313 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:31.802437 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:31.805461 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:31.805465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:31.805466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:31.805481 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:31.807767 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:31.808214 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:31.808217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:31.808218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:31.808329 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:31.808602 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:31.808603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:31.808604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:31.808920 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:31.809075 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:31.809203 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 00:36:31.809352 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":66,"threads_started":1}
I20260808 00:36:31.851497 26327 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:31.852089 26327 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:31.855221 26327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540: Generated new TSK 0
I20260808 00:36:31.855423 26327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:36:31.896044 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46415
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:31.994123 26346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:31.994336 26346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:31.994360 26346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:31.995530 26346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:31.995581 26346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:31.995609 26346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:36:31.997180 26346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:31.997462 26346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:31.997735 26346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:32.003091 26357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:32.003225 26346 server_base.cc:1034] running on GCE node
W20260808 00:36:32.003057 26355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:32.003141 26354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:32.003518 26346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:32.003742 26346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:32.004891 26346 hybrid_clock.cc:648] HybridClock initialized: now 1786149392004872 us; error 35 us; skew 500 ppm
Aug 08 00:36:32 dist-test-slave-44d2 krb5kdc[26191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149392, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:32.007584 26346 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:36:32.007979 26346 webserver.cc:460] Webserver started at http://127.31.210.193:42559/ using document root <none> and password file <none>
I20260808 00:36:32.008194 26346 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:32.008236 26346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:32.008327 26346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:32.009394 26346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "c977b9b93d1e403c861a39e3c0498596"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "7dbc522acd061c921c9a9a7bd79aaae3"
server_key_iv: "9221ce0e963b0ef8c5382c9f99ec131d"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.009714 26346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "c977b9b93d1e403c861a39e3c0498596"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "7dbc522acd061c921c9a9a7bd79aaae3"
server_key_iv: "9221ce0e963b0ef8c5382c9f99ec131d"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.010982 26346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:36:32.011768 26364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.011963 26346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:32.012113 26346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "c977b9b93d1e403c861a39e3c0498596"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "7dbc522acd061c921c9a9a7bd79aaae3"
server_key_iv: "9221ce0e963b0ef8c5382c9f99ec131d"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.012193 26346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:32.034708 26346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:32.036825 26346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:32.037041 26346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:32.037314 26346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:32.037657 26346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:32.037714 26346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.037750 26346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:32.037794 26346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.043815 26346 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:45589
I20260808 00:36:32.043920 26530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:45589 every 8 connection(s)
I20260808 00:36:32.044250 26346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:36:32 dist-test-slave-44d2 krb5kdc[26191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149392, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:32.053659 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 26346
I20260808 00:36:32.053791 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:36:32.054072 32587 external_mini_cluster.cc:1428] Setting key 57967800e72c36b836b0b051fdb080c9
I20260808 00:36:32.054147 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.045170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:38791 (local address 127.31.210.254:46415)
0808 00:36:32.045247 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.045249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.045414 (+ 165us) server_negotiation.cc:407] Connection header received
0808 00:36:32.045870 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.045872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.045883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.045944 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:32.046936 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.047483 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.048569 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.048819 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.048912 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.051129 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:32.051130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:32.051130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:32.051138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:32.052751 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.053138 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.053139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.053140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.053196 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.053418 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.053419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.053419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.053528 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:32.053580 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.053962 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.054055 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:36:32.054384 26534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.045036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46415 (local address 127.31.210.193:38791)
0808 00:36:32.045216 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.045259 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.045755 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.046017 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.046020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.046629 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:32.046829 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.046834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.047599 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.047599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.048409 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.048411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.048919 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.048920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.049070 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.049596 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:32.049604 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:32.051019 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:32.052829 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:32.052837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:32.052839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:32.053071 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:32.053258 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:32.053259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:32.053259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:32.053361 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:32.053597 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:32.053606 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:32.053793 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.053915 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.054135 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
I20260808 00:36:32.054925 26531 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46415
I20260808 00:36:32.055087 26531 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:32.055305 26531 heartbeater.cc:511] Master 127.31.210.254:46415 requested a full tablet report, sending...
I20260808 00:36:32.055897 26255 ts_manager.cc:194] Registered new tserver with Master: c977b9b93d1e403c861a39e3c0498596 (127.31.210.193:45589)
I20260808 00:36:32.056794 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:38791
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:36:32.084151 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46415
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:32.172346 26538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:32.172673 26538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:32.172725 26538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:32.173942 26538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:32.174015 26538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:32.174063 26538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:36:32.175627 26538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:32.175907 26538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:32.176141 26538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:32.181519 26546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:32.181540 26545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:32.181571 26549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:32.181898 26538 server_base.cc:1034] running on GCE node
I20260808 00:36:32.182147 26538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:32.182389 26538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:32.183544 26538 hybrid_clock.cc:648] HybridClock initialized: now 1786149392183524 us; error 40 us; skew 500 ppm
Aug 08 00:36:32 dist-test-slave-44d2 krb5kdc[26191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149392, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:32.186472 26538 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:36:32.187031 26538 webserver.cc:460] Webserver started at http://127.31.210.194:44081/ using document root <none> and password file <none>
I20260808 00:36:32.187299 26538 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:32.187372 26538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:32.187536 26538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:32.188712 26538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "962a3251b4f64884a04541e5b71eddeb"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "1dc511f9f06f3638f94036563338460d"
server_key_iv: "c78a86cf3d993a3e02e631ea7ec4c487"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.189100 26538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "962a3251b4f64884a04541e5b71eddeb"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "1dc511f9f06f3638f94036563338460d"
server_key_iv: "c78a86cf3d993a3e02e631ea7ec4c487"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.190522 26538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:36:32.191345 26558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.191529 26538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:32.191641 26538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "962a3251b4f64884a04541e5b71eddeb"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "1dc511f9f06f3638f94036563338460d"
server_key_iv: "c78a86cf3d993a3e02e631ea7ec4c487"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.191720 26538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:32.208874 26538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:32.211359 26538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:32.211534 26538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:32.211796 26538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:32.212128 26538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:32.212185 26538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.212232 26538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:32.212266 26538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.219004 26538 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:40743
I20260808 00:36:32.219065 26718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:40743 every 8 connection(s)
I20260808 00:36:32.219388 26538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:36:32.220546 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 26538
I20260808 00:36:32.220688 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:36:32.220906 32587 external_mini_cluster.cc:1428] Setting key 37ef3bd3da451c12d36a1c7c19126c27
Aug 08 00:36:32 dist-test-slave-44d2 krb5kdc[26191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149392, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:32.230806 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.220171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:38485 (local address 127.31.210.254:46415)
0808 00:36:32.220222 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.220224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.220561 (+ 337us) server_negotiation.cc:407] Connection header received
0808 00:36:32.221243 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.221245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.221257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.221296 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:32.222536 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.223093 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.224402 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.224670 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.224759 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.227318 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:32.227321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:32.227321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:32.227331 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:32.229162 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.229660 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.229662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.229663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.229744 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.230026 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.230027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.230027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.230186 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:32.230266 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.230581 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.230681 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:36:32.230999 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.220145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46415 (local address 127.31.210.194:38485)
0808 00:36:32.220305 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.220376 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.221117 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.221376 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.221380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.222152 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:32.222431 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.222438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.223192 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.223194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.224308 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.224311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.224771 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.224772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.224917 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.225469 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:32.225478 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:32.227219 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:32.229267 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:32.229275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:32.229278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:32.229571 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:32.229822 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:32.229823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:32.229823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:32.229944 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:32.230260 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:32.230266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:32.230429 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.230529 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.230686 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
I20260808 00:36:32.231662 26719 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46415
I20260808 00:36:32.231789 26719 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:32.232002 26719 heartbeater.cc:511] Master 127.31.210.254:46415 requested a full tablet report, sending...
I20260808 00:36:32.232498 26255 ts_manager.cc:194] Registered new tserver with Master: 962a3251b4f64884a04541e5b71eddeb (127.31.210.194:40743)
I20260808 00:36:32.232954 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:38485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:36:32.252920 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46415
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:32.344475 26725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:32.344743 26725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:32.344767 26725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:32.345924 26725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:32.345963 26725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:32.345989 26725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:36:32.347379 26725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:32.347569 26725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:32.347751 26725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:32.352943 26733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:32.352943 26732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:32.352962 26739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:32.353152 26725 server_base.cc:1034] running on GCE node
I20260808 00:36:32.353410 26725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:32.353665 26725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:32.354822 26725 hybrid_clock.cc:648] HybridClock initialized: now 1786149392354812 us; error 32 us; skew 500 ppm
Aug 08 00:36:32 dist-test-slave-44d2 krb5kdc[26191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149392, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:32.357664 26725 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:36:32.358156 26725 webserver.cc:460] Webserver started at http://127.31.210.195:43949/ using document root <none> and password file <none>
I20260808 00:36:32.358412 26725 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:32.358484 26725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:32.358644 26725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:32.359812 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "ed04e309fd7e4693a9af3c38915d69f6"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "17206eb7d24cfb9b27d16617e557e6b1"
server_key_iv: "1b696750c10178b5955920ba2cd1d5dc"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.360211 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "ed04e309fd7e4693a9af3c38915d69f6"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "17206eb7d24cfb9b27d16617e557e6b1"
server_key_iv: "1b696750c10178b5955920ba2cd1d5dc"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.361760 26725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:36:32.362540 26749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.362736 26725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:32.362843 26725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "ed04e309fd7e4693a9af3c38915d69f6"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "17206eb7d24cfb9b27d16617e557e6b1"
server_key_iv: "1b696750c10178b5955920ba2cd1d5dc"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.362924 26725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:32.377182 26725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:32.379719 26725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:32.379906 26725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:32.380195 26725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:32.380581 26725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:32.380621 26725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.380645 26725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:32.380684 26725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.386633 26725 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:37347
I20260808 00:36:32.386710 26916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:37347 every 8 connection(s)
I20260808 00:36:32.387058 26725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:36:32.389334 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 26725
I20260808 00:36:32.389456 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:36:32.389629 32587 external_mini_cluster.cc:1428] Setting key 3d0a449df866d1b10dfb4c3dcf7dcc9b
Aug 08 00:36:32 dist-test-slave-44d2 krb5kdc[26191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149392, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:32.396973 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.387874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:43021 (local address 127.31.210.254:46415)
0808 00:36:32.387936 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.387938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.388216 (+ 278us) server_negotiation.cc:407] Connection header received
0808 00:36:32.388809 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.388811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.388822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.388865 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:32.389962 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.390706 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.391666 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.391848 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.391898 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.394107 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:32.394108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:32.394109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:32.394115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:32.395599 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.396055 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.396056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.396056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.396112 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.396380 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.396381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.396381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.396525 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:32.396602 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.396835 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.396903 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:36:32.397166 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.387841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46415 (local address 127.31.210.195:43021)
0808 00:36:32.388085 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.388123 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.388703 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.388927 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.388930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.389587 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:32.389870 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.389879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.390786 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.390787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.391596 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.391598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.391915 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.391916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.392061 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.392595 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:32.392612 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:32.394017 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:32.395713 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:32.395724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:32.395726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:32.395985 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:32.396213 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:32.396214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:32.396215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:32.396324 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:32.396594 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:32.396599 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:32.396723 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.396810 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.396938 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":109,"threads_started":1}
I20260808 00:36:32.397646 26917 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46415
I20260808 00:36:32.397773 26917 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:32.397980 26917 heartbeater.cc:511] Master 127.31.210.254:46415 requested a full tablet report, sending...
I20260808 00:36:32.398483 26255 ts_manager.cc:194] Registered new tserver with Master: ed04e309fd7e4693a9af3c38915d69f6 (127.31.210.195:37347)
I20260808 00:36:32.398851 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:43021
I20260808 00:36:32.401243 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:36:32.408969 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.402651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.31.210.254:46415)
0808 00:36:32.402702 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.402704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.402830 (+ 126us) server_negotiation.cc:407] Connection header received
0808 00:36:32.402877 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.402877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.402886 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.402935 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:32.403545 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.404121 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.405024 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.405204 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.405256 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.406617 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:32.406618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:32.406619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:32.406625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:32.407807 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.408196 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.408197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.408197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.408250 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:32.408433 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:32.408433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:32.408433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:32.408651 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:32.408742 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.408851 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.408905 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:36:32.410682 26255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:36:32.412542 26255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:36:32.419350 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.415488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.31.210.194:40743)
0808 00:36:32.415675 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.415680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.415691 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:36:32.415734 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.415736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.415752 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.415836 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:32.415945 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.417175 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.418414 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.418918 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.419008 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.419070 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.419228 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.419269 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260808 00:36:32.419339 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.415082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:40743 (local address 127.0.0.1:52578)
0808 00:36:32.415457 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.415466 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.415508 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.415845 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.415846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.415848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:32.415885 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.415886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.417290 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.417291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.418333 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.418336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.419008 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.419010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.419104 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.419123 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.419196 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.419246 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260808 00:36:32.419498 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.415120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:45589 (local address 127.0.0.1:53888)
0808 00:36:32.415527 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.415536 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.415592 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.416034 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.416035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.416037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:32.416087 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.416090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.417461 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.417462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.418367 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.418386 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.419294 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.419295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.419347 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.419350 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.419400 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.419431 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":79,"threads_started":1}
I20260808 00:36:32.419525 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.415269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.31.210.195:37347)
0808 00:36:32.415485 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.415490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.415526 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:36:32.415572 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.415574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.415593 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.415676 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:32.415887 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.416776 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.418194 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.418935 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.419302 (+ 367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.419359 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.419384 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.419430 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
I20260808 00:36:32.419579 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.415050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:37347 (local address 127.0.0.1:56484)
0808 00:36:32.415233 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.415274 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.415346 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.415696 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.415700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.415702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:32.415777 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.415779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.416873 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.416874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.418095 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.418098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.419041 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.419042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.419109 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.419134 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.419169 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.419231 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
I20260808 00:36:32.419821 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.415334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.31.210.193:45589)
0808 00:36:32.415708 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.415711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.415720 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:32.415756 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.415758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.415775 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.416175 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:32.416280 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.417365 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.418454 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.419147 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.419211 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.419259 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.419701 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.419744 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":94,"threads_started":1}
I20260808 00:36:32.420604 26432 tablet_service.cc:1467] Processing CreateTablet for tablet a21733d3b8ed460ba2194283bd107bbc (DEFAULT_TABLE table=test-table [id=78fb3bfa5725483c8afedcdacdabcb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:32.420615 26825 tablet_service.cc:1467] Processing CreateTablet for tablet a21733d3b8ed460ba2194283bd107bbc (DEFAULT_TABLE table=test-table [id=78fb3bfa5725483c8afedcdacdabcb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:32.420982 26432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21733d3b8ed460ba2194283bd107bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:32.420989 26825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21733d3b8ed460ba2194283bd107bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:32.423573 26942 tablet_bootstrap.cc:492] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Bootstrap starting.
I20260808 00:36:32.423981 26945 tablet_bootstrap.cc:492] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: Bootstrap starting.
I20260808 00:36:32.424214 26628 tablet_service.cc:1467] Processing CreateTablet for tablet a21733d3b8ed460ba2194283bd107bbc (DEFAULT_TABLE table=test-table [id=78fb3bfa5725483c8afedcdacdabcb7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:32.424448 26942 tablet_bootstrap.cc:654] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:32.424575 26628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21733d3b8ed460ba2194283bd107bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:32.424697 26945 tablet_bootstrap.cc:654] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:32.425086 26942 log.cc:824] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:32.425338 26945 log.cc:824] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:32.425943 26942 tablet_bootstrap.cc:492] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: No bootstrap required, opened a new log
I20260808 00:36:32.425992 26942 ts_tablet_manager.cc:1397] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:32.426105 26945 tablet_bootstrap.cc:492] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: No bootstrap required, opened a new log
I20260808 00:36:32.426165 26945 ts_tablet_manager.cc:1397] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:32.427254 26942 raft_consensus.cc:348] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.427378 26942 raft_consensus.cc:374] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:32.427399 26942 raft_consensus.cc:729] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed04e309fd7e4693a9af3c38915d69f6, State: Initialized, Role: FOLLOWER
I20260808 00:36:32.427512 26942 consensus_queue.cc:260] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.427639 26950 tablet_bootstrap.cc:492] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Bootstrap starting.
I20260808 00:36:32.427801 26942 ts_tablet_manager.cc:1428] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:32.428020 26917 heartbeater.cc:503] Master 127.31.210.254:46415 was elected leader, sending a full tablet report...
I20260808 00:36:32.428216 26945 raft_consensus.cc:348] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.428370 26945 raft_consensus.cc:374] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:32.428433 26945 raft_consensus.cc:729] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c977b9b93d1e403c861a39e3c0498596, State: Initialized, Role: FOLLOWER
I20260808 00:36:32.428537 26950 tablet_bootstrap.cc:654] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:32.428603 26945 consensus_queue.cc:260] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.428893 26945 ts_tablet_manager.cc:1428] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:36:32.428903 26531 heartbeater.cc:503] Master 127.31.210.254:46415 was elected leader, sending a full tablet report...
I20260808 00:36:32.429143 26950 log.cc:824] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:32.429905 26950 tablet_bootstrap.cc:492] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: No bootstrap required, opened a new log
I20260808 00:36:32.430001 26950 ts_tablet_manager.cc:1397] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:36:32.431316 26950 raft_consensus.cc:348] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.431470 26950 raft_consensus.cc:374] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:32.431519 26950 raft_consensus.cc:729] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962a3251b4f64884a04541e5b71eddeb, State: Initialized, Role: FOLLOWER
I20260808 00:36:32.431666 26950 consensus_queue.cc:260] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.431962 26950 ts_tablet_manager.cc:1428] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:36:32.431985 26719 heartbeater.cc:503] Master 127.31.210.254:46415 was elected leader, sending a full tablet report...
W20260808 00:36:32.470067 26720 tablet.cc:2369] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:32.545120 26532 tablet.cc:2369] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:32.637674 26918 tablet.cc:2369] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:36:32.659878 26953 raft_consensus.cc:482] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:32.660035 26953 raft_consensus.cc:504] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.660517 26953 leader_election.cc:288] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed04e309fd7e4693a9af3c38915d69f6 (127.31.210.195:37347), 962a3251b4f64884a04541e5b71eddeb (127.31.210.194:40743)
I20260808 00:36:32.663803 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.661022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:34333 (local address 127.31.210.194:40743)
0808 00:36:32.661079 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.661081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.661232 (+ 151us) server_negotiation.cc:407] Connection header received
0808 00:36:32.661313 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.661314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.661323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.661381 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:32.661544 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.662015 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.663073 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.663449 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.663531 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.663540 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.663695 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.663732 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:36:32.663820 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.660913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:40743 (local address 127.31.210.193:34333)
0808 00:36:32.661157 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.661168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.661251 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.661385 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.661386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.661388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:32.661466 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.661468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.662107 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.662108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.662980 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.662982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.663543 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.663544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.663625 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.663629 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.663691 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.663732 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":18,"thread_start_us":65,"threads_started":1}
I20260808 00:36:32.663899 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.661086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:45597 (local address 127.31.210.195:37347)
0808 00:36:32.661152 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.661155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.661163 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:36:32.661194 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.661195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.661204 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.661253 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:32.661453 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.662128 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.663151 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.663538 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.663625 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.663634 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.663786 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.663827 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:36:32.663923 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.660684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:37347 (local address 127.31.210.193:45597)
0808 00:36:32.660816 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.660829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:32.660888 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:32.661307 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:32.661308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:32.661311 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:32.661383 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.661385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.662240 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.662241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.663052 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:32.663054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.663624 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.663625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:32.663695 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.663699 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:32.663777 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 00:36:32.663821 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260808 00:36:32.664336 26657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21733d3b8ed460ba2194283bd107bbc" candidate_uuid: "c977b9b93d1e403c861a39e3c0498596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962a3251b4f64884a04541e5b71eddeb" is_pre_election: true
I20260808 00:36:32.664363 26853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21733d3b8ed460ba2194283bd107bbc" candidate_uuid: "c977b9b93d1e403c861a39e3c0498596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed04e309fd7e4693a9af3c38915d69f6" is_pre_election: true
I20260808 00:36:32.664485 26657 raft_consensus.cc:2454] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c977b9b93d1e403c861a39e3c0498596 in term 0.
I20260808 00:36:32.664543 26853 raft_consensus.cc:2454] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c977b9b93d1e403c861a39e3c0498596 in term 0.
I20260808 00:36:32.664764 26367 leader_election.cc:302] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962a3251b4f64884a04541e5b71eddeb, c977b9b93d1e403c861a39e3c0498596; no voters:
I20260808 00:36:32.664963 26953 raft_consensus.cc:2781] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:36:32.665014 26953 raft_consensus.cc:482] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:36:32.665069 26953 raft_consensus.cc:3037] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:32.665859 26953 raft_consensus.cc:504] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.666020 26953 leader_election.cc:288] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [CANDIDATE]: Term 1 election: Requested vote from peers ed04e309fd7e4693a9af3c38915d69f6 (127.31.210.195:37347), 962a3251b4f64884a04541e5b71eddeb (127.31.210.194:40743)
I20260808 00:36:32.666189 26853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21733d3b8ed460ba2194283bd107bbc" candidate_uuid: "c977b9b93d1e403c861a39e3c0498596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed04e309fd7e4693a9af3c38915d69f6"
I20260808 00:36:32.666205 26657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21733d3b8ed460ba2194283bd107bbc" candidate_uuid: "c977b9b93d1e403c861a39e3c0498596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962a3251b4f64884a04541e5b71eddeb"
I20260808 00:36:32.666273 26853 raft_consensus.cc:3037] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:32.666273 26657 raft_consensus.cc:3037] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:32.667192 26657 raft_consensus.cc:2454] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c977b9b93d1e403c861a39e3c0498596 in term 1.
I20260808 00:36:32.667191 26853 raft_consensus.cc:2454] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c977b9b93d1e403c861a39e3c0498596 in term 1.
I20260808 00:36:32.667397 26367 leader_election.cc:302] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 962a3251b4f64884a04541e5b71eddeb, c977b9b93d1e403c861a39e3c0498596, ed04e309fd7e4693a9af3c38915d69f6; no voters:
I20260808 00:36:32.667507 26953 raft_consensus.cc:2781] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:32.667680 26953 raft_consensus.cc:686] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 1 LEADER]: Becoming Leader. State: Replica: c977b9b93d1e403c861a39e3c0498596, State: Running, Role: LEADER
I20260808 00:36:32.667783 26953 consensus_queue.cc:237] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } }
I20260808 00:36:32.668702 26256 catalog_manager.cc:5495] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 reported cstate change: term changed from 0 to 1, leader changed from <none> to c977b9b93d1e403c861a39e3c0498596 (127.31.210.193). New cstate: current_term: 1 leader_uuid: "c977b9b93d1e403c861a39e3c0498596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c977b9b93d1e403c861a39e3c0498596" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 45589 } health_report { overall_health: HEALTHY } } }
I20260808 00:36:32.703828 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.700847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.31.210.193:45589)
0808 00:36:32.700910 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.700913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.700922 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:32.701105 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.701106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.701127 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.701204 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:36:32.701391 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.701888 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.702904 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.703118 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.703177 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.703344 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:36:32.703483 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:36:32.703565 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.703678 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.703706 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:36:32.706141 26657 raft_consensus.cc:1261] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 1 FOLLOWER]: Refusing update from remote peer c977b9b93d1e403c861a39e3c0498596: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:32.706151 26853 raft_consensus.cc:1261] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 1 FOLLOWER]: Refusing update from remote peer c977b9b93d1e403c861a39e3c0498596: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:32.706555 26953 consensus_queue.cc:1035] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "962a3251b4f64884a04541e5b71eddeb" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:32.706748 26960 consensus_queue.cc:1035] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed04e309fd7e4693a9af3c38915d69f6" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 37347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:32.709846 26966 mvcc.cc:205] Tried to move back new op lower bound from 7316067912521101312 to 7316067912367845376. Current Snapshot: MvccSnapshot[applied={T|T < 7316067912521101312}]
I20260808 00:36:32.710120 26965 mvcc.cc:205] Tried to move back new op lower bound from 7316067912521101312 to 7316067912367845376. Current Snapshot: MvccSnapshot[applied={T|T < 7316067912521101312}]
I20260808 00:36:32.712991 26256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:32.713119 26256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:32.713934 26256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 796944ac1d91475fbfc31e3b62ac5540: Sending DeleteTablet for 3 replicas of tablet a21733d3b8ed460ba2194283bd107bbc
I20260808 00:36:32.714306 26628 tablet_service.cc:1514] Processing DeleteTablet for tablet a21733d3b8ed460ba2194283bd107bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:32 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52578
I20260808 00:36:32.714320 26432 tablet_service.cc:1514] Processing DeleteTablet for tablet a21733d3b8ed460ba2194283bd107bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:32 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:53888
I20260808 00:36:32.714532 26825 tablet_service.cc:1514] Processing DeleteTablet for tablet a21733d3b8ed460ba2194283bd107bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:32 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:56484
I20260808 00:36:32.714685 26976 tablet_replica.cc:330] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596: stopping tablet replica
I20260808 00:36:32.714789 26976 raft_consensus.cc:2229] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:36:32.714797 26975 tablet_replica.cc:330] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: stopping tablet replica
I20260808 00:36:32.714898 26975 raft_consensus.cc:2229] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:32.714964 26976 raft_consensus.cc:2258] T a21733d3b8ed460ba2194283bd107bbc P c977b9b93d1e403c861a39e3c0498596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:32.715123 26975 raft_consensus.cc:2258] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:32.715207 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 26346
I20260808 00:36:32.717870 26977 tablet_replica.cc:330] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: stopping tablet replica
I20260808 00:36:32.717983 26977 raft_consensus.cc:2229] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:32.718077 26977 raft_consensus.cc:2258] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:32.719625 26977 ts_tablet_manager.cc:1905] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:32.719906 26975 ts_tablet_manager.cc:1905] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:32.721659 26977 ts_tablet_manager.cc:1918] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:32.721738 26977 log.cc:1196] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/a21733d3b8ed460ba2194283bd107bbc
I20260808 00:36:32.721938 26975 ts_tablet_manager.cc:1918] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:32.722002 26975 log.cc:1196] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/a21733d3b8ed460ba2194283bd107bbc
I20260808 00:36:32.722064 26977 ts_tablet_manager.cc:1939] T a21733d3b8ed460ba2194283bd107bbc P ed04e309fd7e4693a9af3c38915d69f6: Deleting consensus metadata
I20260808 00:36:32.722218 26975 ts_tablet_manager.cc:1939] T a21733d3b8ed460ba2194283bd107bbc P 962a3251b4f64884a04541e5b71eddeb: Deleting consensus metadata
I20260808 00:36:32.722684 26235 catalog_manager.cc:4841] TS 962a3251b4f64884a04541e5b71eddeb (127.31.210.194:40743): tablet a21733d3b8ed460ba2194283bd107bbc (table test-table [id=78fb3bfa5725483c8afedcdacdabcb7f]) successfully deleted
I20260808 00:36:32.723438 26234 catalog_manager.cc:4841] TS ed04e309fd7e4693a9af3c38915d69f6 (127.31.210.195:37347): tablet a21733d3b8ed460ba2194283bd107bbc (table test-table [id=78fb3bfa5725483c8afedcdacdabcb7f]) successfully deleted
W20260808 00:36:32.729771 26233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:45589 (error 108)
I20260808 00:36:32.730109 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 26538
I20260808 00:36:32.731783 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.730243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:45589 (local address 127.0.0.1:53910)
0808 00:36:32.731678 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:32.731736 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1406}
W20260808 00:36:32.734634 26233 catalog_manager.cc:4584] TS c977b9b93d1e403c861a39e3c0498596 (127.31.210.193:45589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a21733d3b8ed460ba2194283bd107bbc: Network error: Client connection negotiation failed: client connection to 127.31.210.193:45589: connect: Connection refused (error 111)
I20260808 00:36:32.745607 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 26725
I20260808 00:36:32.755595 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 26213
2026-08-08T00:36:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2524 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T00:36:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:32Z Disabled control of system clock
I20260808 00:36:32.790373 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46573
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:46573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:32.876437 26982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:32.876693 26982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:32.876713 26982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:32.877875 26982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:32.877925 26982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:32.877938 26982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:32.877951 26982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:32.878005 26982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:32.879336 26982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:46573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:32.879606 26982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:32.879922 26982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:32.884917 26992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:32.885128 26990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:32.885035 26994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:32.885151 26982 server_base.cc:1034] running on GCE node
I20260808 00:36:32.885524 26982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:32.885824 26982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:32.886987 26982 hybrid_clock.cc:648] HybridClock initialized: now 1786149392886970 us; error 38 us; skew 500 ppm
I20260808 00:36:32.888230 26982 webserver.cc:460] Webserver started at http://127.31.210.254:42395/ using document root <none> and password file <none>
I20260808 00:36:32.888525 26982 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:32.888650 26982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:32.888816 26982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:32.889866 26982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "1c9b0d12834d49b48c8a19a7a1c666c5"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "d6df907fcca74e542095ad80fa37cd15"
server_key_iv: "a0465868a2d5da2f2576f0662fecc97a"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.890247 26982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "1c9b0d12834d49b48c8a19a7a1c666c5"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "d6df907fcca74e542095ad80fa37cd15"
server_key_iv: "a0465868a2d5da2f2576f0662fecc97a"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.891536 26982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:32.892241 27000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:32.892409 26982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:32.892530 26982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "1c9b0d12834d49b48c8a19a7a1c666c5"
format_stamp: "Formatted at 2026-08-08 00:36:32 on dist-test-slave-44d2"
server_key: "d6df907fcca74e542095ad80fa37cd15"
server_key_iv: "a0465868a2d5da2f2576f0662fecc97a"
server_key_version: "encryptionkey@0"
I20260808 00:36:32.892617 26982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:32.908567 26982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:32.908910 26982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:32.909065 26982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:32.913163 26982 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:46573
I20260808 00:36:32.913208 27085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:46573 every 8 connection(s)
I20260808 00:36:32.913583 26982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:32.914574 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:32.915546 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 26982
I20260808 00:36:32.915632 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:32.915848 32587 external_mini_cluster.cc:1428] Setting key fcf5ba55e68d647e0abf87aad01de73f
I20260808 00:36:32.917538 27087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5: Bootstrap starting.
I20260808 00:36:32.918150 27087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:32.918458 27087 log.cc:824] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:32.919163 27087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5: No bootstrap required, opened a new log
I20260808 00:36:32.921042 27087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46573 } }
I20260808 00:36:32.921206 27087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:32.921271 27087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9b0d12834d49b48c8a19a7a1c666c5, State: Initialized, Role: FOLLOWER
I20260808 00:36:32.921476 27087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46573 } }
I20260808 00:36:32.921613 27087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:32.921656 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:32.916391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.31.210.254:46573)
0808 00:36:32.916623 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:32.916632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:32.916666 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:36:32.917223 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:32.917234 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:32.917316 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:32.917464 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:32.917728 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.918767 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.919605 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:32.920023 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:32.920185 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:32.920936 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:32.920940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:32.920940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:32.921129 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:32.921163 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:32.921175 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:32.921253 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:32.921334 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 00:36:32.921419 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
I20260808 00:36:32.921661 27087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:32.921766 27087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:32.922348 27087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46573 } }
I20260808 00:36:32.922459 27087 leader_election.cc:302] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c9b0d12834d49b48c8a19a7a1c666c5; no voters:
I20260808 00:36:32.922624 27087 leader_election.cc:288] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:32.922673 27098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:32.922821 27098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [term 1 LEADER]: Becoming Leader. State: Replica: 1c9b0d12834d49b48c8a19a7a1c666c5, State: Running, Role: LEADER
I20260808 00:36:32.922919 27087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:32.922956 27098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46573 } }
I20260808 00:36:32.923396 27099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46573 } } }
I20260808 00:36:32.923396 27100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c9b0d12834d49b48c8a19a7a1c666c5. Latest consensus state: current_term: 1 leader_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c9b0d12834d49b48c8a19a7a1c666c5" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 46573 } } }
I20260808 00:36:32.923533 27099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:32.923619 27100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:32.924031 27108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:32.924580 27108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:32.926385 27108 catalog_manager.cc:1347] Generated new cluster ID: dcb8ac46a5de47e6a6412d66fa9c9969
I20260808 00:36:32.926443 27108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:32.955157 27108 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:32.955735 27108 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:32.960156 27108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5: Generated new TSK 0
I20260808 00:36:32.960338 27108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:32.966741 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46573
--builtin_ntp_servers=127.31.210.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:33.060182 27124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:33.060417 27124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:33.060442 27124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:33.061703 27124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:33.061755 27124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:33.061782 27124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:36:33.063270 27124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:33.063526 27124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:33.063848 27124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:33.069180 27134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:33.069196 27140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:33.069237 27135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:33.069372 27124 server_base.cc:1034] running on GCE node
I20260808 00:36:33.069639 27124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:33.069860 27124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:33.070995 27124 hybrid_clock.cc:648] HybridClock initialized: now 1786149393070981 us; error 32 us; skew 500 ppm
I20260808 00:36:33.072366 27124 webserver.cc:460] Webserver started at http://127.31.210.193:38529/ using document root <none> and password file <none>
I20260808 00:36:33.072649 27124 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:33.072700 27124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:33.072795 27124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:33.073921 27124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "897de9191b05c2a77f620f5fc04e9332"
server_key_iv: "f851188bbb9285959b1f2ad897de13d0"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.074258 27124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "897de9191b05c2a77f620f5fc04e9332"
server_key_iv: "f851188bbb9285959b1f2ad897de13d0"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.075527 27124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:33.076251 27148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.076454 27124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:33.076566 27124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "897de9191b05c2a77f620f5fc04e9332"
server_key_iv: "f851188bbb9285959b1f2ad897de13d0"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.076627 27124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:33.086756 27124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:33.087021 27124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:33.087123 27124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:33.087298 27124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:33.087587 27124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:33.087618 27124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.087638 27124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:33.087651 27124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.093431 27124 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:34435
I20260808 00:36:33.093467 27314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:34435 every 8 connection(s)
I20260808 00:36:33.093842 27124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:36:33.099428 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.094641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:35561 (local address 127.31.210.254:46573)
0808 00:36:33.094714 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.094716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.095015 (+ 299us) server_negotiation.cc:407] Connection header received
0808 00:36:33.095491 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.095492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.095503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.095572 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:33.096336 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.097012 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.097993 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.098178 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.098230 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.099182 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:33.099183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:33.099183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:33.099188 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:33.099203 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:33.099206 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:33.099251 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.099337 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.099362 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:36:33.099709 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.094593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46573 (local address 127.31.210.193:35561)
0808 00:36:33.094814 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.094856 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.095379 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.095690 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.095694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.095704 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:33.096245 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.096250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.097103 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.097104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.097914 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.097916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.098252 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.098253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.098367 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.099013 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:33.099014 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:33.099046 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:33.099076 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:33.099264 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:33.099269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:33.099272 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.099326 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.099415 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260808 00:36:33.100241 27315 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46573
I20260808 00:36:33.100358 27315 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:33.100625 27315 heartbeater.cc:511] Master 127.31.210.254:46573 requested a full tablet report, sending...
I20260808 00:36:33.101167 27035 ts_manager.cc:194] Registered new tserver with Master: adbc37b7f9b84ad9bd0109f88a5bae5b (127.31.210.193:34435)
I20260808 00:36:33.101814 27035 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.193:35561
I20260808 00:36:33.103713 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 27124
I20260808 00:36:33.103832 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:36:33.104056 32587 external_mini_cluster.cc:1428] Setting key a357c333312fe88d55482575ea64b918
I20260808 00:36:33.105032 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46573
--builtin_ntp_servers=127.31.210.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:33.189229 27322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:33.189484 27322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:33.189517 27322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:33.190896 27322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:33.190958 27322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:33.190987 27322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:36:33.192595 27322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:33.192891 27322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:33.193215 27322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:33.198884 27332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:33.198918 27330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:33.198921 27329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:33.199055 27322 server_base.cc:1034] running on GCE node
I20260808 00:36:33.199330 27322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:33.199538 27322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:33.200680 27322 hybrid_clock.cc:648] HybridClock initialized: now 1786149393200658 us; error 41 us; skew 500 ppm
I20260808 00:36:33.201999 27322 webserver.cc:460] Webserver started at http://127.31.210.194:33715/ using document root <none> and password file <none>
I20260808 00:36:33.202260 27322 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:33.202319 27322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:33.202490 27322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:33.203585 27322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "83f89a5405204e17baea13920ebaa149"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "59939a0c1f889817968e6e55ec7d9d85"
server_key_iv: "c9b63f0ac8e737a332ca78a425b82997"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.203953 27322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "83f89a5405204e17baea13920ebaa149"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "59939a0c1f889817968e6e55ec7d9d85"
server_key_iv: "c9b63f0ac8e737a332ca78a425b82997"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.205355 27322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:36:33.206092 27341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.206277 27322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:36:33.206365 27322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "83f89a5405204e17baea13920ebaa149"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "59939a0c1f889817968e6e55ec7d9d85"
server_key_iv: "c9b63f0ac8e737a332ca78a425b82997"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.206422 27322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:33.227988 27322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:33.228304 27322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:33.228412 27322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:33.228698 27322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:33.229063 27322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:33.229099 27322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.229140 27322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:33.229168 27322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.234790 27322 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:39755
I20260808 00:36:33.234840 27511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:39755 every 8 connection(s)
I20260808 00:36:33.235231 27322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:36:33.240553 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.236010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:39273 (local address 127.31.210.254:46573)
0808 00:36:33.236092 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.236094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.236341 (+ 247us) server_negotiation.cc:407] Connection header received
0808 00:36:33.236928 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.236929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.236939 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.236996 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:33.237695 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.238204 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.239195 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.239410 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.239464 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.240282 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:33.240283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:33.240284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:33.240288 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:33.240303 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:33.240307 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:33.240349 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.240432 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.240455 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:36:33.240788 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.236009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46573 (local address 127.31.210.194:39273)
0808 00:36:33.236209 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.236251 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.236838 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.237063 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.237067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.237075 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:33.237610 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.237616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.238297 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.238299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.239117 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.239120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.239490 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.239502 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.239618 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.240149 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:33.240150 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:33.240179 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:33.240202 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:33.240360 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:33.240364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:33.240367 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.240420 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.240572 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260808 00:36:33.241010 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 27322
I20260808 00:36:33.241158 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:36:33.241346 27512 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46573
I20260808 00:36:33.241428 32587 external_mini_cluster.cc:1428] Setting key 73b9b02635a2b23dbca4447fc657b7af
I20260808 00:36:33.241451 27512 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:33.241672 27512 heartbeater.cc:511] Master 127.31.210.254:46573 requested a full tablet report, sending...
I20260808 00:36:33.242038 27034 ts_manager.cc:194] Registered new tserver with Master: 83f89a5405204e17baea13920ebaa149 (127.31.210.194:39755)
I20260808 00:36:33.242432 27034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.194:39273
I20260808 00:36:33.242436 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46573
--builtin_ntp_servers=127.31.210.212:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:33.336166 27518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:33.336400 27518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:33.336429 27518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:33.337690 27518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:33.337750 27518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:33.337778 27518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:36:33.339306 27518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:33.339612 27518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:33.339967 27518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:33.345418 27529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:33.345430 27534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:33.345441 27531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:33.345453 27518 server_base.cc:1034] running on GCE node
I20260808 00:36:33.345932 27518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:33.346210 27518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:33.347383 27518 hybrid_clock.cc:648] HybridClock initialized: now 1786149393347366 us; error 34 us; skew 500 ppm
I20260808 00:36:33.348882 27518 webserver.cc:460] Webserver started at http://127.31.210.195:33187/ using document root <none> and password file <none>
I20260808 00:36:33.349210 27518 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:33.349262 27518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:33.349428 27518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:33.350581 27518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "79a3230370ec42118cbe8584e2267b9d"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "007e03097e38ae9979c09f524e8833e6"
server_key_iv: "7ad6bfd758f26310334ecefa5fe23a98"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.350957 27518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "79a3230370ec42118cbe8584e2267b9d"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "007e03097e38ae9979c09f524e8833e6"
server_key_iv: "7ad6bfd758f26310334ecefa5fe23a98"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.352455 27518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:33.353341 27543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.353521 27518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:36:33.353618 27518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "79a3230370ec42118cbe8584e2267b9d"
format_stamp: "Formatted at 2026-08-08 00:36:33 on dist-test-slave-44d2"
server_key: "007e03097e38ae9979c09f524e8833e6"
server_key_iv: "7ad6bfd758f26310334ecefa5fe23a98"
server_key_version: "encryptionkey@0"
I20260808 00:36:33.353705 27518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:33.401434 27518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:33.401777 27518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:33.401937 27518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:33.402194 27518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:33.402552 27518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:33.402607 27518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.402649 27518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:33.402673 27518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:33.408859 27518 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:38873
I20260808 00:36:33.408912 27724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:38873 every 8 connection(s)
I20260808 00:36:33.409360 27518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:36:33.414898 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.410164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:49007 (local address 127.31.210.254:46573)
0808 00:36:33.410216 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.410219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.410541 (+ 322us) server_negotiation.cc:407] Connection header received
0808 00:36:33.411092 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.411093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.411106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.411162 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:33.411891 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.412406 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.413423 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.413623 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.413702 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.414581 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:33.414583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:33.414583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:33.414588 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:33.414602 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:33.414606 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:33.414669 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.414787 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.414817 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:36:33.415123 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.410182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:46573 (local address 127.31.210.195:49007)
0808 00:36:33.410412 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.410459 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.410984 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.411235 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.411240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.411251 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:33.411794 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.411798 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.412522 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.412523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.413331 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.413333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.413709 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.413711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.413849 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.414417 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:33.414417 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:33.414447 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:33.414471 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:33.414671 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:33.414677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:33.414681 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.414786 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.414892 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260808 00:36:33.415689 27725 heartbeater.cc:347] Connected to a master server at 127.31.210.254:46573
I20260808 00:36:33.415793 27725 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:33.415978 27725 heartbeater.cc:511] Master 127.31.210.254:46573 requested a full tablet report, sending...
I20260808 00:36:33.416366 27034 ts_manager.cc:194] Registered new tserver with Master: 79a3230370ec42118cbe8584e2267b9d (127.31.210.195:38873)
I20260808 00:36:33.416790 27034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.195:49007
I20260808 00:36:33.419644 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 27518
I20260808 00:36:33.419755 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:36:33.419977 32587 external_mini_cluster.cc:1428] Setting key 2a542923541284b353eab57864a219cc
I20260808 00:36:33.421007 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:36:33.425828 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.422447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.31.210.254:46573)
0808 00:36:33.422489 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.422491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.422641 (+ 150us) server_negotiation.cc:407] Connection header received
0808 00:36:33.422722 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.422723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.422734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.422779 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:33.422952 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.423567 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.424515 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.424717 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.424790 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.425530 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:33.425532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:33.425532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:33.425537 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:33.425554 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:33.425558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:33.425660 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.425729 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.425740 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:36:33.427695 27034 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:36:33.429353 27034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:36:33.435864 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.431729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41596 (local address 127.31.210.194:39755)
0808 00:36:33.432081 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.432086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.432097 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:36:33.432144 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.432146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.432161 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.432462 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:33.432549 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.433661 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.434817 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.435529 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.435600 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.435679 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.435741 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.435777 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":77,"threads_started":1}
I20260808 00:36:33.435873 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.431467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:39755 (local address 127.0.0.1:41596)
0808 00:36:33.431831 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.431843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.431905 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.432286 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.432289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.432291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:33.432379 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.432382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.433749 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.433750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.434735 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.434737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.435599 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.435600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.435661 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.435665 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.435715 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.435760 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":4,"thread_start_us":53,"threads_started":1}
I20260808 00:36:33.435946 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.431631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.31.210.195:38873)
0808 00:36:33.431836 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.431841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.432012 (+ 171us) server_negotiation.cc:407] Connection header received
0808 00:36:33.432098 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.432100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.432115 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.432191 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:33.432503 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.433239 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.434580 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.435297 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.435377 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.435469 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.435798 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.435841 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260808 00:36:33.435722 27739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.431484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:38873 (local address 127.0.0.1:52492)
0808 00:36:33.431952 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.431960 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.432015 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.432285 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.432288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.432294 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:33.432395 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.432398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.433326 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.433328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.434487 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.434489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.435384 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.435385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.435462 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.435477 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.435573 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.435640 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":28,"thread_start_us":58,"threads_started":1}
I20260808 00:36:33.436722 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.431467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:34435 (local address 127.0.0.1:35094)
0808 00:36:33.431735 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.431794 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.431868 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.432325 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.432326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.432328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:33.432382 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.432383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.434803 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.434804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.435558 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.435560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.436452 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.436453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.436538 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.436542 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.436613 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.436653 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":1,"thread_start_us":57,"threads_started":1}
I20260808 00:36:33.436738 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.431909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.31.210.193:34435)
0808 00:36:33.432114 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.432118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.432126 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:36:33.432165 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.432166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.432179 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.432256 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:33.432464 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.433666 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.435647 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.436366 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.436454 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.436551 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.436626 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.436661 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":94,"threads_started":1}
I20260808 00:36:33.436753 27623 tablet_service.cc:1467] Processing CreateTablet for tablet 95fbcaaf2af04688aec0862028ec832c (DEFAULT_TABLE table=test-table [id=16f86ec56f42488c9f41b7b419942131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:33.436991 27409 tablet_service.cc:1467] Processing CreateTablet for tablet 95fbcaaf2af04688aec0862028ec832c (DEFAULT_TABLE table=test-table [id=16f86ec56f42488c9f41b7b419942131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:33.437153 27623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95fbcaaf2af04688aec0862028ec832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:33.437301 27409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95fbcaaf2af04688aec0862028ec832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:33.440727 27747 tablet_bootstrap.cc:492] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Bootstrap starting.
I20260808 00:36:33.440727 27750 tablet_bootstrap.cc:492] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Bootstrap starting.
I20260808 00:36:33.441315 27213 tablet_service.cc:1467] Processing CreateTablet for tablet 95fbcaaf2af04688aec0862028ec832c (DEFAULT_TABLE table=test-table [id=16f86ec56f42488c9f41b7b419942131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:33.441485 27750 tablet_bootstrap.cc:654] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:33.441485 27747 tablet_bootstrap.cc:654] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:33.441702 27213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95fbcaaf2af04688aec0862028ec832c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:33.441830 27747 log.cc:824] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:33.441830 27750 log.cc:824] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:33.442538 27750 tablet_bootstrap.cc:492] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: No bootstrap required, opened a new log
I20260808 00:36:33.442605 27747 tablet_bootstrap.cc:492] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: No bootstrap required, opened a new log
I20260808 00:36:33.442620 27750 ts_tablet_manager.cc:1397] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:36:33.442674 27747 ts_tablet_manager.cc:1397] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:36:33.444581 27750 raft_consensus.cc:348] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.444741 27750 raft_consensus.cc:374] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:33.444783 27750 raft_consensus.cc:729] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a3230370ec42118cbe8584e2267b9d, State: Initialized, Role: FOLLOWER
I20260808 00:36:33.444893 27750 consensus_queue.cc:260] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.445194 27750 ts_tablet_manager.cc:1428] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:36:33.445204 27725 heartbeater.cc:503] Master 127.31.210.254:46573 was elected leader, sending a full tablet report...
I20260808 00:36:33.445286 27747 raft_consensus.cc:348] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.445457 27747 raft_consensus.cc:374] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:33.445516 27747 raft_consensus.cc:729] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f89a5405204e17baea13920ebaa149, State: Initialized, Role: FOLLOWER
I20260808 00:36:33.445510 27755 tablet_bootstrap.cc:492] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b: Bootstrap starting.
I20260808 00:36:33.445655 27747 consensus_queue.cc:260] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.445928 27747 ts_tablet_manager.cc:1428] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:36:33.445989 27512 heartbeater.cc:503] Master 127.31.210.254:46573 was elected leader, sending a full tablet report...
I20260808 00:36:33.446179 27755 tablet_bootstrap.cc:654] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:33.446542 27755 log.cc:824] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:33.447284 27755 tablet_bootstrap.cc:492] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b: No bootstrap required, opened a new log
I20260808 00:36:33.447341 27755 ts_tablet_manager.cc:1397] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:36:33.449175 27755 raft_consensus.cc:348] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.449296 27755 raft_consensus.cc:374] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:33.449317 27755 raft_consensus.cc:729] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adbc37b7f9b84ad9bd0109f88a5bae5b, State: Initialized, Role: FOLLOWER
I20260808 00:36:33.449421 27755 consensus_queue.cc:260] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.449636 27755 ts_tablet_manager.cc:1428] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:36:33.449718 27315 heartbeater.cc:503] Master 127.31.210.254:46573 was elected leader, sending a full tablet report...
W20260808 00:36:33.485833 27514 tablet.cc:2369] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:36:33.513549 27759 raft_consensus.cc:482] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:33.513685 27759 raft_consensus.cc:504] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.514142 27759 leader_election.cc:288] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79a3230370ec42118cbe8584e2267b9d (127.31.210.195:38873), adbc37b7f9b84ad9bd0109f88a5bae5b (127.31.210.193:34435)
I20260808 00:36:33.517222 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.514302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:34435 (local address 127.31.210.194:40305)
0808 00:36:33.514375 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.514386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.514445 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.514604 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.514606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.514607 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:33.514703 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.514705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.515439 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.515440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.516255 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.516257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.516922 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.516923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.517033 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.517038 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.517098 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.517133 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 00:36:33.517241 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.514395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:40305 (local address 127.31.210.193:34435)
0808 00:36:33.514445 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.514447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.514484 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:36:33.514516 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.514517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.514527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.514594 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:33.514778 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.515357 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.516361 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.516839 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.516898 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.516905 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.517115 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.517168 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:36:33.517333 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.514360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:47615 (local address 127.31.210.195:38873)
0808 00:36:33.514415 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.514417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.514723 (+ 306us) server_negotiation.cc:407] Connection header received
0808 00:36:33.514760 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.514761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.514773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.514826 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:33.514996 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.515538 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.516561 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.516951 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.517027 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.517036 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.517208 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.517258 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:36:33.517369 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.514306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:38873 (local address 127.31.210.194:47615)
0808 00:36:33.514626 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.514635 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:33.514685 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:33.514848 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:33.514849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:33.514852 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:33.514923 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.514924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.515649 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.515650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.516441 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:33.516444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.517036 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.517037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:33.517127 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.517131 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:33.517189 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:36:33.517224 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":103,"thread_start_us":154,"threads_started":1}
I20260808 00:36:33.517613 27245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fbcaaf2af04688aec0862028ec832c" candidate_uuid: "83f89a5405204e17baea13920ebaa149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" is_pre_election: true
I20260808 00:36:33.517656 27651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fbcaaf2af04688aec0862028ec832c" candidate_uuid: "83f89a5405204e17baea13920ebaa149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a3230370ec42118cbe8584e2267b9d" is_pre_election: true
I20260808 00:36:33.517783 27651 raft_consensus.cc:2454] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f89a5405204e17baea13920ebaa149 in term 0.
I20260808 00:36:33.517784 27245 raft_consensus.cc:2454] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f89a5405204e17baea13920ebaa149 in term 0.
I20260808 00:36:33.518062 27342 leader_election.cc:302] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a3230370ec42118cbe8584e2267b9d, 83f89a5405204e17baea13920ebaa149; no voters:
I20260808 00:36:33.518206 27759 raft_consensus.cc:2781] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:36:33.518270 27759 raft_consensus.cc:482] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:36:33.518286 27759 raft_consensus.cc:3037] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:33.519075 27759 raft_consensus.cc:504] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.519243 27759 leader_election.cc:288] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [CANDIDATE]: Term 1 election: Requested vote from peers 79a3230370ec42118cbe8584e2267b9d (127.31.210.195:38873), adbc37b7f9b84ad9bd0109f88a5bae5b (127.31.210.193:34435)
I20260808 00:36:33.519381 27651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fbcaaf2af04688aec0862028ec832c" candidate_uuid: "83f89a5405204e17baea13920ebaa149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a3230370ec42118cbe8584e2267b9d"
I20260808 00:36:33.519433 27245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95fbcaaf2af04688aec0862028ec832c" candidate_uuid: "83f89a5405204e17baea13920ebaa149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b"
I20260808 00:36:33.519461 27651 raft_consensus.cc:3037] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:33.519502 27245 raft_consensus.cc:3037] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:33.520404 27651 raft_consensus.cc:2454] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f89a5405204e17baea13920ebaa149 in term 1.
I20260808 00:36:33.520467 27245 raft_consensus.cc:2454] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f89a5405204e17baea13920ebaa149 in term 1.
I20260808 00:36:33.520591 27342 leader_election.cc:302] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a3230370ec42118cbe8584e2267b9d, 83f89a5405204e17baea13920ebaa149; no voters:
I20260808 00:36:33.520709 27759 raft_consensus.cc:2781] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:33.520905 27759 raft_consensus.cc:686] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 1 LEADER]: Becoming Leader. State: Replica: 83f89a5405204e17baea13920ebaa149, State: Running, Role: LEADER
I20260808 00:36:33.521032 27759 consensus_queue.cc:237] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } }
I20260808 00:36:33.521929 27034 catalog_manager.cc:5495] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f89a5405204e17baea13920ebaa149 (127.31.210.194). New cstate: current_term: 1 leader_uuid: "83f89a5405204e17baea13920ebaa149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83f89a5405204e17baea13920ebaa149" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 39755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260808 00:36:33.532761 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.530490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.31.210.194:39755)
0808 00:36:33.530547 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:33.530549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:33.530558 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:33.530601 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:33.530602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:33.530619 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:33.530679 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:36:33.530790 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.531252 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.531941 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:33.532132 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:33.532186 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:33.532322 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:36:33.532409 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:36:33.532526 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:33.532621 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 00:36:33.532646 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:36:33.534826 27245 raft_consensus.cc:1261] T 95fbcaaf2af04688aec0862028ec832c P adbc37b7f9b84ad9bd0109f88a5bae5b [term 1 FOLLOWER]: Refusing update from remote peer 83f89a5405204e17baea13920ebaa149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:33.535120 27651 raft_consensus.cc:1261] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 1 FOLLOWER]: Refusing update from remote peer 83f89a5405204e17baea13920ebaa149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:33.535243 27766 consensus_queue.cc:1035] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adbc37b7f9b84ad9bd0109f88a5bae5b" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:33.535392 27759 consensus_queue.cc:1035] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a3230370ec42118cbe8584e2267b9d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 38873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:33.538861 27771 mvcc.cc:205] Tried to move back new op lower bound from 7316067915914866688 to 7316067915862814720. Current Snapshot: MvccSnapshot[applied={T|T < 7316067915914866688}]
I20260808 00:36:33.540441 27034 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:33.540587 27034 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:33.541368 27034 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c9b0d12834d49b48c8a19a7a1c666c5: Sending DeleteTablet for 3 replicas of tablet 95fbcaaf2af04688aec0862028ec832c
I20260808 00:36:33.541736 27409 tablet_service.cc:1514] Processing DeleteTablet for tablet 95fbcaaf2af04688aec0862028ec832c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:33 UTC) from {username='slave'} at 127.0.0.1:41596
I20260808 00:36:33.541745 27213 tablet_service.cc:1514] Processing DeleteTablet for tablet 95fbcaaf2af04688aec0862028ec832c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:33 UTC) from {username='slave'} at 127.0.0.1:35094
I20260808 00:36:33.541821 27623 tablet_service.cc:1514] Processing DeleteTablet for tablet 95fbcaaf2af04688aec0862028ec832c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:33 UTC) from {username='slave'} at 127.0.0.1:52492
I20260808 00:36:33.542003 27785 tablet_replica.cc:330] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: stopping tablet replica
I20260808 00:36:33.542098 27786 tablet_replica.cc:330] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: stopping tablet replica
I20260808 00:36:33.542186 27786 raft_consensus.cc:2229] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:33.542187 27785 raft_consensus.cc:2229] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:36:33.542299 27786 raft_consensus.cc:2258] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:33.542357 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 27124
I20260808 00:36:33.542990 27785 raft_consensus.cc:2258] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:33.543310 27786 ts_tablet_manager.cc:1905] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:33.545243 27786 ts_tablet_manager.cc:1918] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:33.545320 27786 log.cc:1196] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/95fbcaaf2af04688aec0862028ec832c
I20260808 00:36:33.545596 27786 ts_tablet_manager.cc:1939] T 95fbcaaf2af04688aec0862028ec832c P 79a3230370ec42118cbe8584e2267b9d: Deleting consensus metadata
I20260808 00:36:33.545866 27785 ts_tablet_manager.cc:1905] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:33.547253 27785 ts_tablet_manager.cc:1918] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:33.547343 27785 log.cc:1196] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/95fbcaaf2af04688aec0862028ec832c
I20260808 00:36:33.547636 27785 ts_tablet_manager.cc:1939] T 95fbcaaf2af04688aec0862028ec832c P 83f89a5405204e17baea13920ebaa149: Deleting consensus metadata
I20260808 00:36:33.548048 27002 catalog_manager.cc:4841] TS 83f89a5405204e17baea13920ebaa149 (127.31.210.194:39755): tablet 95fbcaaf2af04688aec0862028ec832c (table test-table [id=16f86ec56f42488c9f41b7b419942131]) successfully deleted
I20260808 00:36:33.549844 27001 catalog_manager.cc:4841] TS 79a3230370ec42118cbe8584e2267b9d (127.31.210.195:38873): tablet 95fbcaaf2af04688aec0862028ec832c (table test-table [id=16f86ec56f42488c9f41b7b419942131]) successfully deleted
W20260808 00:36:33.555820 27003 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:34435 (error 108)
I20260808 00:36:33.556147 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 27322
I20260808 00:36:33.556344 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:33.556213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:34435 (local address 127.0.0.1:35100)
0808 00:36:33.556273 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:33.556309 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 00:36:33.556595 27003 catalog_manager.cc:4584] TS adbc37b7f9b84ad9bd0109f88a5bae5b (127.31.210.193:34435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95fbcaaf2af04688aec0862028ec832c: Network error: Client connection negotiation failed: client connection to 127.31.210.193:34435: connect: Connection refused (error 111)
I20260808 00:36:33.565104 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 27518
I20260808 00:36:33.575361 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 26982
2026-08-08T00:36:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (823 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:33 dist-test-slave-44d2 krb5kdc[27792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:33 dist-test-slave-44d2 krb5kdc[27792](info): set up 2 sockets
Aug 08 00:36:33 dist-test-slave-44d2 krb5kdc[27792](info): commencing operation
krb5kdc: starting...
W20260808 00:36:34.833139 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:34 dist-test-slave-44d2 krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:36:34.968716 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:40007
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:40007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:35.057520 27812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:35.057844 27812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:35.057889 27812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:35.059059 27812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:35.059098 27812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:35.059113 27812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:35.059125 27812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:35.059139 27812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:35.060598 27812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:40007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:40007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:35.060807 27812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:35.060987 27812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:35.065970 27821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:35.066167 27820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:35.066222 27812 server_base.cc:1034] running on GCE node
W20260808 00:36:35.066285 27824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:35.066504 27812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:35.066789 27812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:35.067931 27812 hybrid_clock.cc:648] HybridClock initialized: now 1786149395067915 us; error 34 us; skew 500 ppm
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:35.070567 27812 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:36:35.071055 27812 webserver.cc:460] Webserver started at http://127.31.210.254:32939/ using document root <none> and password file <none>
I20260808 00:36:35.071277 27812 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:35.071323 27812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:35.071483 27812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:35.072698 27812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "349b8846bfcb4081ac32ab9bb72b14c7"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "6073cae261ef93b9815acb0e27e7a4b5"
server_key_iv: "d5b3a77238f115be8dcdbec7219e8136"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.073071 27812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "349b8846bfcb4081ac32ab9bb72b14c7"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "6073cae261ef93b9815acb0e27e7a4b5"
server_key_iv: "d5b3a77238f115be8dcdbec7219e8136"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.074406 27812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:36:35.075187 27834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.075354 27812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:35.075453 27812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "349b8846bfcb4081ac32ab9bb72b14c7"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "6073cae261ef93b9815acb0e27e7a4b5"
server_key_iv: "d5b3a77238f115be8dcdbec7219e8136"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.075532 27812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:35.091277 27812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:35.093464 27812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:35.093638 27812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:35.098081 27812 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:40007
I20260808 00:36:35.098121 27907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:40007 every 8 connection(s)
I20260808 00:36:35.098541 27812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:35.099537 27908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:35.101994 27908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7: Bootstrap starting.
I20260808 00:36:35.102626 27908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:35.102926 27908 log.cc:824] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:35.103679 27908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7: No bootstrap required, opened a new log
I20260808 00:36:35.105268 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 27812
I20260808 00:36:35.105373 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:35.105631 32587 external_mini_cluster.cc:1428] Setting key 4a59e0c84bc5b993ab70e1240dcd8e9f
I20260808 00:36:35.105525 27908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40007 } }
I20260808 00:36:35.105685 27908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:35.105751 27908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349b8846bfcb4081ac32ab9bb72b14c7, State: Initialized, Role: FOLLOWER
I20260808 00:36:35.105922 27908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40007 } }
I20260808 00:36:35.106002 27908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:35.106065 27908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:35.106120 27908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:35.107028 27908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40007 } }
I20260808 00:36:35.107151 27908 leader_election.cc:302] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 349b8846bfcb4081ac32ab9bb72b14c7; no voters:
I20260808 00:36:35.107339 27908 leader_election.cc:288] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:35.107360 27915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:35.107476 27915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [term 1 LEADER]: Becoming Leader. State: Replica: 349b8846bfcb4081ac32ab9bb72b14c7, State: Running, Role: LEADER
I20260808 00:36:35.107630 27908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:35.107620 27915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40007 } }
I20260808 00:36:35.108165 27916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 349b8846bfcb4081ac32ab9bb72b14c7. Latest consensus state: current_term: 1 leader_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40007 } } }
I20260808 00:36:35.108261 27916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:35.108748 27917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349b8846bfcb4081ac32ab9bb72b14c7" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 40007 } } }
I20260808 00:36:35.109391 27917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7 [sys.catalog]: This master's current role is: LEADER
W20260808 00:36:35.109645 27941 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:36:35.109704 27941 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:36:35.109747 27924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:35.109891 27924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:35.111341 27924 catalog_manager.cc:1347] Generated new cluster ID: 1b74838341de4ed7b7a29705fd6d0525
I20260808 00:36:35.111398 27924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:35.121903 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.106205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.31.210.254:40007)
0808 00:36:35.106396 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.106408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.106448 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:36:35.106577 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.106582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.106627 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.106756 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:35.107544 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.108926 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.110072 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.110697 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.110810 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.118500 (+ 7690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:35.118504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:35.118505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:35.118518 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:35.120216 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.120661 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.120664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.120664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.120756 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.120980 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.120981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.120982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.121227 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:35.121396 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.121495 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.121643 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260808 00:36:35.122357 27924 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:35.122952 27924 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:35.133318 27924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7: Generated new TSK 0
I20260808 00:36:35.133554 27924 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.210.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
Entry for principal kudu/127.31.210.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab.
I20260808 00:36:35.164032 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40007
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:35.249131 27948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:35.249344 27948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:35.249368 27948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:35.250583 27948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:35.250633 27948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:35.250660 27948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:36:35.252159 27948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.193.keytab
--principal=kudu/127.31.210.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:35.252429 27948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:35.252743 27948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:35.258026 27958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:35.258157 27948 server_base.cc:1034] running on GCE node
W20260808 00:36:35.258033 27955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:35.258126 27954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:35.258529 27948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:35.258783 27948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:35.259941 27948 hybrid_clock.cc:648] HybridClock initialized: now 1786149395259926 us; error 27 us; skew 500 ppm
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:35.262802 27948 init.cc:377] Logged in from keytab as kudu/127.31.210.193@KRBTEST.COM (short username kudu)
I20260808 00:36:35.263414 27948 webserver.cc:460] Webserver started at http://127.31.210.193:36361/ using document root <none> and password file <none>
I20260808 00:36:35.263736 27948 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:35.263787 27948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:35.263937 27948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:35.265105 27948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "e5da289f902e4d4f8d7b279cea3bd222"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "363f069ff60020e372e8a8c740292aec"
server_key_iv: "7701c83dc4f001ee0b7e4474bd25bd5e"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.265486 27948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "e5da289f902e4d4f8d7b279cea3bd222"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "363f069ff60020e372e8a8c740292aec"
server_key_iv: "7701c83dc4f001ee0b7e4474bd25bd5e"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.266892 27948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:36:35.267671 27969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.267840 27948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:35.267944 27948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "e5da289f902e4d4f8d7b279cea3bd222"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "363f069ff60020e372e8a8c740292aec"
server_key_iv: "7701c83dc4f001ee0b7e4474bd25bd5e"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.268069 27948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:35.302767 27948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:35.305450 27948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:35.305620 27948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:35.305863 27948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:35.306201 27948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:35.306254 27948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.306296 27948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:35.306321 27948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.312626 27948 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:39467
I20260808 00:36:35.312680 28140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:39467 every 8 connection(s)
I20260808 00:36:35.313015 27948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.193@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:35.320595 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 27948
I20260808 00:36:35.320731 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:36:35.320979 32587 external_mini_cluster.cc:1428] Setting key 1c152cb5dc2a0ac958c282ed6a0300c6
I20260808 00:36:35.322942 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.313764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:54299 (local address 127.31.210.254:40007)
0808 00:36:35.313857 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.313859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.314024 (+ 165us) server_negotiation.cc:407] Connection header received
0808 00:36:35.314499 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.314500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.314517 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.314593 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:35.315565 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.316062 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.317056 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.317253 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.317305 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.319846 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:35.319847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:35.319848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:35.319854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:35.321444 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.321951 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.321952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.321953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.322026 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.322293 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.322294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.322294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.322414 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:35.322481 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.322763 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.322851 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 00:36:35.323160 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.313727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:40007 (local address 127.31.210.193:54299)
0808 00:36:35.313897 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.313934 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.314416 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.314648 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.314652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.315225 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:35.315470 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.315475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.316145 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.316146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.316973 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.316975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.317325 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.317326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.317437 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.317983 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:35.318028 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:35.319738 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:35.321566 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:35.321574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:35.321577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:35.321860 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:35.322097 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:35.322098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:35.322099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:35.322219 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:35.322484 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:35.322492 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:35.322633 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.322740 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.322903 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260808 00:36:35.324023 28143 heartbeater.cc:347] Connected to a master server at 127.31.210.254:40007
I20260808 00:36:35.324147 28143 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:35.324359 28143 heartbeater.cc:511] Master 127.31.210.254:40007 requested a full tablet report, sending...
I20260808 00:36:35.324988 27857 ts_manager.cc:194] Registered new tserver with Master: e5da289f902e4d4f8d7b279cea3bd222 (127.31.210.193:39467)
I20260808 00:36:35.325626 27857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.193@KRBTEST.COM'} at 127.31.210.193:54299
WARNING: no policy specified for kudu/127.31.210.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
Entry for principal kudu/127.31.210.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab.
I20260808 00:36:35.351759 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40007
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:35.435671 28149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:35.435921 28149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:35.435961 28149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:35.437153 28149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:35.437227 28149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:35.437269 28149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:36:35.438719 28149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.194.keytab
--principal=kudu/127.31.210.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:35.438989 28149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:35.439234 28149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:35.444290 28159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:35.444296 28157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:35.444296 28162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:35.444612 28149 server_base.cc:1034] running on GCE node
I20260808 00:36:35.444837 28149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:35.445071 28149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:35.446216 28149 hybrid_clock.cc:648] HybridClock initialized: now 1786149395446203 us; error 30 us; skew 500 ppm
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:35.448839 28149 init.cc:377] Logged in from keytab as kudu/127.31.210.194@KRBTEST.COM (short username kudu)
I20260808 00:36:35.449290 28149 webserver.cc:460] Webserver started at http://127.31.210.194:37201/ using document root <none> and password file <none>
I20260808 00:36:35.449514 28149 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:35.449579 28149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:35.449726 28149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:35.450799 28149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "39eb4d5c95b0475a84c653a00a223205"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "d7feca260967fbac262b5e6663818d5e"
server_key_iv: "5a9863b7aae9e2fc24b0cafd3b9daaa1"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.451169 28149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "39eb4d5c95b0475a84c653a00a223205"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "d7feca260967fbac262b5e6663818d5e"
server_key_iv: "5a9863b7aae9e2fc24b0cafd3b9daaa1"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.452559 28149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:35.453361 28170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.453549 28149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:35.453647 28149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "39eb4d5c95b0475a84c653a00a223205"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "d7feca260967fbac262b5e6663818d5e"
server_key_iv: "5a9863b7aae9e2fc24b0cafd3b9daaa1"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.453727 28149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:35.471882 28149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:35.473948 28149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:35.474118 28149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:35.474380 28149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:35.474704 28149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:35.474738 28149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.474778 28149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:35.474807 28149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.481078 28149 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:36517
I20260808 00:36:35.481174 28331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:36517 every 8 connection(s)
I20260808 00:36:35.481477 28149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.194@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:35.487715 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 28149
I20260808 00:36:35.487859 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:36:35.488070 32587 external_mini_cluster.cc:1428] Setting key fdd4e00c234dd1860c01744c49aba774
I20260808 00:36:35.491073 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.482215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:40895 (local address 127.31.210.254:40007)
0808 00:36:35.482291 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.482292 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.482508 (+ 216us) server_negotiation.cc:407] Connection header received
0808 00:36:35.482985 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.482986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.482995 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.483070 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:35.483972 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.484465 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.485493 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.485688 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.485743 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.487868 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:35.487870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:35.487870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:35.487876 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:35.489651 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.490069 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.490070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.490070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.490136 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.490374 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.490375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.490375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.490529 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:35.490598 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.490875 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.490962 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:36:35.491248 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.482210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:40007 (local address 127.31.210.194:40895)
0808 00:36:35.482377 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.482421 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.482901 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.483125 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.483128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.483693 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:35.483885 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.483890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.484595 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.484597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.485420 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.485423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.485759 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.485760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.485856 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.486309 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:35.486318 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:35.487728 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:35.489740 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:35.489747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:35.489748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:35.489992 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:35.490205 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:35.490206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:35.490206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:35.490312 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:35.490592 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:35.490597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:35.490711 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.490848 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.491005 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260808 00:36:35.491820 28332 heartbeater.cc:347] Connected to a master server at 127.31.210.254:40007
I20260808 00:36:35.491921 28332 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:35.492107 28332 heartbeater.cc:511] Master 127.31.210.254:40007 requested a full tablet report, sending...
I20260808 00:36:35.492527 27857 ts_manager.cc:194] Registered new tserver with Master: 39eb4d5c95b0475a84c653a00a223205 (127.31.210.194:36517)
I20260808 00:36:35.493058 27857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.194@KRBTEST.COM'} at 127.31.210.194:40895
WARNING: no policy specified for kudu/127.31.210.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
Entry for principal kudu/127.31.210.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab.
I20260808 00:36:35.519193 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40007
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:40577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:35.604810 28337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:35.605027 28337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:35.605053 28337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:35.606176 28337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:35.606215 28337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:35.606240 28337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:36:35.607722 28337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:40577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.195.keytab
--principal=kudu/127.31.210.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:35.607923 28337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:35.608193 28337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:35.613415 28344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:35.613415 28345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:35.613518 28348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:35.613895 28337 server_base.cc:1034] running on GCE node
I20260808 00:36:35.614166 28337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:35.614408 28337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:35.615564 28337 hybrid_clock.cc:648] HybridClock initialized: now 1786149395615550 us; error 28 us; skew 500 ppm
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:35.618227 28337 init.cc:377] Logged in from keytab as kudu/127.31.210.195@KRBTEST.COM (short username kudu)
I20260808 00:36:35.618675 28337 webserver.cc:460] Webserver started at http://127.31.210.195:42379/ using document root <none> and password file <none>
I20260808 00:36:35.618907 28337 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:35.618996 28337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:35.619163 28337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:35.620231 28337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "8b9cc22624de4231840741b5c21197bf"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "a926d0b62d41f3513e9184a7b7778b6a"
server_key_iv: "8de737008de423b0862ec64787ae1f8c"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.620641 28337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "8b9cc22624de4231840741b5c21197bf"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "a926d0b62d41f3513e9184a7b7778b6a"
server_key_iv: "8de737008de423b0862ec64787ae1f8c"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.622007 28337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:35.622834 28360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.623034 28337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:35.623142 28337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "8b9cc22624de4231840741b5c21197bf"
format_stamp: "Formatted at 2026-08-08 00:36:35 on dist-test-slave-44d2"
server_key: "a926d0b62d41f3513e9184a7b7778b6a"
server_key_iv: "8de737008de423b0862ec64787ae1f8c"
server_key_version: "encryptionkey@0"
I20260808 00:36:35.623222 28337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:35.656287 28337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:35.658428 28337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:35.658603 28337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:35.658857 28337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:35.659253 28337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:35.659313 28337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.659358 28337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:35.659397 28337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:35.665452 28337 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:34005
I20260808 00:36:35.665503 28540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:34005 every 8 connection(s)
I20260808 00:36:35.665889 28337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:36:35 dist-test-slave-44d2 krb5kdc[27792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149395, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.195@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:35.675187 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.666706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:43699 (local address 127.31.210.254:40007)
0808 00:36:35.666789 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.666791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.666955 (+ 164us) server_negotiation.cc:407] Connection header received
0808 00:36:35.667519 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.667520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.667531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.667597 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:35.668662 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.669159 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.670138 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.670321 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.670397 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.672661 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:35.672663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:35.672663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:35.672670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:35.674065 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.674420 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.674421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.674421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.674472 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.674679 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.674679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.674680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.674780 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:35.674830 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.675058 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.675120 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:36:35.675403 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 28337
I20260808 00:36:35.675485 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.666625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:40007 (local address 127.31.210.195:43699)
0808 00:36:35.666803 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.666847 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.667421 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.667662 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.667667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.668318 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:35.668577 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.668584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.669241 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.669242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.670063 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.670065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.670383 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.670384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.670518 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.671005 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:36:35.671014 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:35.672565 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:35.674134 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:35.674140 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:35.674142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:35.674357 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:35.674524 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:36:35.674525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:36:35.674525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:36:35.674627 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:36:35.674831 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:35.674835 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:35.674948 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.675031 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.675221 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260808 00:36:35.675518 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:36:35.675761 32587 external_mini_cluster.cc:1428] Setting key 830cfa9c076bd97b14bbae8d9d5da140
I20260808 00:36:35.676072 28541 heartbeater.cc:347] Connected to a master server at 127.31.210.254:40007
I20260808 00:36:35.676196 28541 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:35.676414 28541 heartbeater.cc:511] Master 127.31.210.254:40007 requested a full tablet report, sending...
I20260808 00:36:35.676844 27857 ts_manager.cc:194] Registered new tserver with Master: 8b9cc22624de4231840741b5c21197bf (127.31.210.195:34005)
I20260808 00:36:35.677232 27857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.210.195@KRBTEST.COM'} at 127.31.210.195:43699
I20260808 00:36:35.687640 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:36:35.695300 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.689124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.31.210.254:40007)
0808 00:36:35.689205 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.689208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.689306 (+ 98us) server_negotiation.cc:407] Connection header received
0808 00:36:35.689388 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.689389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.689400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.689458 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:35.690045 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.690531 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.691267 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.691475 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.691555 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.692822 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:35.692823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:35.692823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:35.692831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:35.694129 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.694460 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.694461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.694462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.694522 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:35.694724 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:35.694725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:35.694726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:35.694885 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:35.694992 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.695109 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.695194 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:36:35.696825 27857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:36:35.698458 27857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:36:35.705461 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.700952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37634 (local address 127.31.210.193:39467)
0808 00:36:35.701119 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.701124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.701443 (+ 319us) server_negotiation.cc:407] Connection header received
0808 00:36:35.701551 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.701554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.701572 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.701655 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:35.701864 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.703054 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.704377 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.705002 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.705089 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.705152 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.705323 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.705370 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260808 00:36:35.705451 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.700833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:39467 (local address 127.0.0.1:37634)
0808 00:36:35.701330 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.701363 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.701456 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.701663 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.701667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.701671 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:35.701745 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.701748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.703164 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.703165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.704267 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.704268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.705086 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.705087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.705183 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.705199 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.705303 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.705368 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":59,"thread_start_us":74,"threads_started":1}
I20260808 00:36:35.705557 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.700846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34005 (local address 127.0.0.1:56218)
0808 00:36:35.701604 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.701614 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.701661 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.701960 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.701961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.701962 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:35.702035 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.702036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.702914 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.702915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.704231 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.704234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.705282 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.705284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.705354 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.705366 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.705420 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.705475 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":8,"thread_start_us":109,"threads_started":1}
I20260808 00:36:35.705725 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.700846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:36517 (local address 127.0.0.1:52312)
0808 00:36:35.701731 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.701739 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.701784 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.702025 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.702025 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.702026 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:35.702056 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.702057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.703314 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.703315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.704233 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.704236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.705284 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.705285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.705354 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.705368 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.705420 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.705475 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":4,"thread_start_us":105,"threads_started":1}
I20260808 00:36:35.705761 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.701145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.31.210.194:36517)
0808 00:36:35.701346 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.701350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.701843 (+ 493us) server_negotiation.cc:407] Connection header received
0808 00:36:35.701904 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.701906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.701924 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.702008 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:35.702120 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.703182 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.704360 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.705112 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.705188 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.705242 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.705523 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.705587 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260808 00:36:35.706148 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.701105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.31.210.195:34005)
0808 00:36:35.701308 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.701312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.701739 (+ 427us) server_negotiation.cc:407] Connection header received
0808 00:36:35.701784 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.701787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.701805 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.701879 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:35.702121 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.702812 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.704326 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.705112 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.705187 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.705242 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.705678 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.705731 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260808 00:36:35.706504 28045 tablet_service.cc:1467] Processing CreateTablet for tablet da277a70d8274edea538084b9a3fade9 (DEFAULT_TABLE table=test-table [id=996cdaff5ca6475d9f4a5edb2e5d7cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:35.706525 28233 tablet_service.cc:1467] Processing CreateTablet for tablet da277a70d8274edea538084b9a3fade9 (DEFAULT_TABLE table=test-table [id=996cdaff5ca6475d9f4a5edb2e5d7cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:35.706884 28045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da277a70d8274edea538084b9a3fade9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:35.706899 28233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da277a70d8274edea538084b9a3fade9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:35.709934 28563 tablet_bootstrap.cc:492] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Bootstrap starting.
I20260808 00:36:35.709935 28562 tablet_bootstrap.cc:492] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222: Bootstrap starting.
I20260808 00:36:35.710630 28563 tablet_bootstrap.cc:654] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:35.710659 28562 tablet_bootstrap.cc:654] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:35.710513 28429 tablet_service.cc:1467] Processing CreateTablet for tablet da277a70d8274edea538084b9a3fade9 (DEFAULT_TABLE table=test-table [id=996cdaff5ca6475d9f4a5edb2e5d7cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:35.710903 28429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da277a70d8274edea538084b9a3fade9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:35.711194 28563 log.cc:824] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:35.711243 28562 log.cc:824] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:35.711988 28562 tablet_bootstrap.cc:492] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222: No bootstrap required, opened a new log
I20260808 00:36:35.712044 28563 tablet_bootstrap.cc:492] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: No bootstrap required, opened a new log
I20260808 00:36:35.712085 28562 ts_tablet_manager.cc:1397] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:35.712131 28563 ts_tablet_manager.cc:1397] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:35.713692 28562 raft_consensus.cc:348] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.713857 28562 raft_consensus.cc:374] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:35.713924 28562 raft_consensus.cc:729] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5da289f902e4d4f8d7b279cea3bd222, State: Initialized, Role: FOLLOWER
I20260808 00:36:35.714100 28562 consensus_queue.cc:260] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.714412 28562 ts_tablet_manager.cc:1428] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:36:35.714296 28563 raft_consensus.cc:348] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.714465 28563 raft_consensus.cc:374] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:35.714514 28563 raft_consensus.cc:729] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39eb4d5c95b0475a84c653a00a223205, State: Initialized, Role: FOLLOWER
I20260808 00:36:35.714519 28567 tablet_bootstrap.cc:492] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Bootstrap starting.
I20260808 00:36:35.714618 28143 heartbeater.cc:503] Master 127.31.210.254:40007 was elected leader, sending a full tablet report...
I20260808 00:36:35.714628 28563 consensus_queue.cc:260] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.714921 28563 ts_tablet_manager.cc:1428] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:36:35.714943 28332 heartbeater.cc:503] Master 127.31.210.254:40007 was elected leader, sending a full tablet report...
I20260808 00:36:35.715265 28567 tablet_bootstrap.cc:654] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:35.715934 28567 log.cc:824] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:35.716735 28567 tablet_bootstrap.cc:492] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: No bootstrap required, opened a new log
I20260808 00:36:35.716830 28567 ts_tablet_manager.cc:1397] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:36:35.718123 28567 raft_consensus.cc:348] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.718250 28567 raft_consensus.cc:374] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:35.718277 28567 raft_consensus.cc:729] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9cc22624de4231840741b5c21197bf, State: Initialized, Role: FOLLOWER
I20260808 00:36:35.718355 28567 consensus_queue.cc:260] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.718554 28567 ts_tablet_manager.cc:1428] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:36:35.718605 28541 heartbeater.cc:503] Master 127.31.210.254:40007 was elected leader, sending a full tablet report...
W20260808 00:36:35.732075 28333 tablet.cc:2369] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:35.813831 28144 tablet.cc:2369] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:35.916618 28542 tablet.cc:2369] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:36:35.936004 28568 raft_consensus.cc:482] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:35.936151 28568 raft_consensus.cc:504] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.936574 28568 leader_election.cc:288] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b9cc22624de4231840741b5c21197bf (127.31.210.195:34005), 39eb4d5c95b0475a84c653a00a223205 (127.31.210.194:36517)
I20260808 00:36:35.939697 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.936913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:47187 (local address 127.31.210.195:34005)
0808 00:36:35.936985 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.936988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.937068 (+ 80us) server_negotiation.cc:407] Connection header received
0808 00:36:35.937153 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.937154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.937164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.937236 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:35.937411 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.937881 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.938913 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.939307 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.939397 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.939407 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.939573 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.939612 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:36:35.939713 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.936913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:36959 (local address 127.31.210.194:36517)
0808 00:36:35.936973 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.936976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.937016 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:36:35.937110 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.937112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.937127 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.937194 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:35.937369 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.937845 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.938917 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.939316 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.939422 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.939432 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.939598 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.939642 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:36:35.939744 28577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.936782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:36517 (local address 127.31.210.193:36959)
0808 00:36:35.936955 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.936968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.937036 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.937202 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.937203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.937205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:35.937285 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.937287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.937930 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.937932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.938811 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.938813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.939432 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.939433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.939506 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.939510 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.939592 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.939640 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":57,"thread_start_us":76,"threads_started":1}
I20260808 00:36:35.939750 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.936782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:34005 (local address 127.31.210.193:47187)
0808 00:36:35.936997 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:35.937008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:35.937089 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:35.937240 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:35.937241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:35.937242 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:35.937343 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.937344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.937967 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.937968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.938813 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:35.938815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.939408 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.939410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:35.939500 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.939504 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:35.939567 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:36:35.939616 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":21,"thread_start_us":72,"threads_started":1}
I20260808 00:36:35.940150 28265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da277a70d8274edea538084b9a3fade9" candidate_uuid: "e5da289f902e4d4f8d7b279cea3bd222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39eb4d5c95b0475a84c653a00a223205" is_pre_election: true
I20260808 00:36:35.940335 28265 raft_consensus.cc:2454] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5da289f902e4d4f8d7b279cea3bd222 in term 0.
I20260808 00:36:35.940297 28467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da277a70d8274edea538084b9a3fade9" candidate_uuid: "e5da289f902e4d4f8d7b279cea3bd222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9cc22624de4231840741b5c21197bf" is_pre_election: true
I20260808 00:36:35.940429 28467 raft_consensus.cc:2454] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5da289f902e4d4f8d7b279cea3bd222 in term 0.
I20260808 00:36:35.940694 27970 leader_election.cc:302] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39eb4d5c95b0475a84c653a00a223205, e5da289f902e4d4f8d7b279cea3bd222; no voters:
I20260808 00:36:35.940876 28568 raft_consensus.cc:2781] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:36:35.940924 28568 raft_consensus.cc:482] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:36:35.940939 28568 raft_consensus.cc:3037] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:35.941663 28568 raft_consensus.cc:504] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.941813 28568 leader_election.cc:288] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [CANDIDATE]: Term 1 election: Requested vote from peers 8b9cc22624de4231840741b5c21197bf (127.31.210.195:34005), 39eb4d5c95b0475a84c653a00a223205 (127.31.210.194:36517)
I20260808 00:36:35.941957 28467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da277a70d8274edea538084b9a3fade9" candidate_uuid: "e5da289f902e4d4f8d7b279cea3bd222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9cc22624de4231840741b5c21197bf"
I20260808 00:36:35.941978 28265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da277a70d8274edea538084b9a3fade9" candidate_uuid: "e5da289f902e4d4f8d7b279cea3bd222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39eb4d5c95b0475a84c653a00a223205"
I20260808 00:36:35.942036 28467 raft_consensus.cc:3037] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:35.942116 28265 raft_consensus.cc:3037] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:35.942792 28467 raft_consensus.cc:2454] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5da289f902e4d4f8d7b279cea3bd222 in term 1.
I20260808 00:36:35.942968 27973 leader_election.cc:302] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b9cc22624de4231840741b5c21197bf, e5da289f902e4d4f8d7b279cea3bd222; no voters:
I20260808 00:36:35.942983 28265 raft_consensus.cc:2454] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5da289f902e4d4f8d7b279cea3bd222 in term 1.
I20260808 00:36:35.943074 28568 raft_consensus.cc:2781] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:35.943249 28568 raft_consensus.cc:686] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [term 1 LEADER]: Becoming Leader. State: Replica: e5da289f902e4d4f8d7b279cea3bd222, State: Running, Role: LEADER
I20260808 00:36:35.943361 28568 consensus_queue.cc:237] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } }
I20260808 00:36:35.944229 27857 catalog_manager.cc:5495] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5da289f902e4d4f8d7b279cea3bd222 (127.31.210.193). New cstate: current_term: 1 leader_uuid: "e5da289f902e4d4f8d7b279cea3bd222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5da289f902e4d4f8d7b279cea3bd222" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 39467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 } health_report { overall_health: UNKNOWN } } }
I20260808 00:36:35.987975 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:35.985622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.31.210.193:39467)
0808 00:36:35.985683 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:35.985685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:35.985694 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:35.985744 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:35.985745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:35.985765 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:35.985837 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:36:35.985956 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.986514 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.987262 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:35.987456 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:35.987513 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:35.987685 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:36:35.987793 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:36:35.987859 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:35.987891 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 00:36:35.987911 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:36:35.990084 28467 raft_consensus.cc:1261] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 1 FOLLOWER]: Refusing update from remote peer e5da289f902e4d4f8d7b279cea3bd222: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:35.990108 28265 raft_consensus.cc:1261] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 1 FOLLOWER]: Refusing update from remote peer e5da289f902e4d4f8d7b279cea3bd222: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:35.990480 28578 consensus_queue.cc:1035] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b9cc22624de4231840741b5c21197bf" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:35.990617 28568 consensus_queue.cc:1035] T da277a70d8274edea538084b9a3fade9 P e5da289f902e4d4f8d7b279cea3bd222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39eb4d5c95b0475a84c653a00a223205" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 36517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:35.993251 28581 mvcc.cc:205] Tried to move back new op lower bound from 7316067925972455424 to 7316067925784670208. Current Snapshot: MvccSnapshot[applied={T|T < 7316067925972455424}]
I20260808 00:36:35.993588 28583 mvcc.cc:205] Tried to move back new op lower bound from 7316067925972455424 to 7316067925784670208. Current Snapshot: MvccSnapshot[applied={T|T < 7316067925972455424}]
I20260808 00:36:35.994230 28585 mvcc.cc:205] Tried to move back new op lower bound from 7316067925972455424 to 7316067925784670208. Current Snapshot: MvccSnapshot[applied={T|T < 7316067925972455424}]
I20260808 00:36:35.995733 27855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:35.995834 27855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:35.996644 27855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 349b8846bfcb4081ac32ab9bb72b14c7: Sending DeleteTablet for 3 replicas of tablet da277a70d8274edea538084b9a3fade9
I20260808 00:36:35.996973 28045 tablet_service.cc:1514] Processing DeleteTablet for tablet da277a70d8274edea538084b9a3fade9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:35 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:37634
I20260808 00:36:35.996987 28233 tablet_service.cc:1514] Processing DeleteTablet for tablet da277a70d8274edea538084b9a3fade9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:35 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:52312
I20260808 00:36:35.997015 28429 tablet_service.cc:1514] Processing DeleteTablet for tablet da277a70d8274edea538084b9a3fade9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:35 UTC) from {username='kudu', principal='kudu/127.31.210.254@KRBTEST.COM'} at 127.0.0.1:56218
I20260808 00:36:35.997231 28597 tablet_replica.cc:330] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: stopping tablet replica
I20260808 00:36:35.997345 28597 raft_consensus.cc:2229] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:35.997432 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 27948
I20260808 00:36:36.007385 28598 tablet_replica.cc:330] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: stopping tablet replica
I20260808 00:36:36.007602 28598 raft_consensus.cc:2229] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:36.007921 28597 raft_consensus.cc:2258] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:36.008526 28597 ts_tablet_manager.cc:1905] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:36.008919 28598 raft_consensus.cc:2258] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:36.009470 28598 ts_tablet_manager.cc:1905] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:36.010409 28597 ts_tablet_manager.cc:1918] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:36.010514 28597 log.cc:1196] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/da277a70d8274edea538084b9a3fade9
I20260808 00:36:36.010780 28597 ts_tablet_manager.cc:1939] T da277a70d8274edea538084b9a3fade9 P 39eb4d5c95b0475a84c653a00a223205: Deleting consensus metadata
I20260808 00:36:36.011257 27836 catalog_manager.cc:4841] TS 39eb4d5c95b0475a84c653a00a223205 (127.31.210.194:36517): tablet da277a70d8274edea538084b9a3fade9 (table test-table [id=996cdaff5ca6475d9f4a5edb2e5d7cfa]) successfully deleted
I20260808 00:36:36.011586 28598 ts_tablet_manager.cc:1918] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:36.011677 28598 log.cc:1196] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/da277a70d8274edea538084b9a3fade9
I20260808 00:36:36.011945 28598 ts_tablet_manager.cc:1939] T da277a70d8274edea538084b9a3fade9 P 8b9cc22624de4231840741b5c21197bf: Deleting consensus metadata
I20260808 00:36:36.012512 27838 catalog_manager.cc:4841] TS 8b9cc22624de4231840741b5c21197bf (127.31.210.195:34005): tablet da277a70d8274edea538084b9a3fade9 (table test-table [id=996cdaff5ca6475d9f4a5edb2e5d7cfa]) successfully deleted
W20260808 00:36:36.013656 27837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:39467 (error 108)
I20260808 00:36:36.013837 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 28149
I20260808 00:36:36.017103 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.016941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:39467 (local address 127.0.0.1:37656)
0808 00:36:36.017007 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:36.017058 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:39467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 00:36:36.017261 27837 catalog_manager.cc:4584] TS e5da289f902e4d4f8d7b279cea3bd222 (127.31.210.193:39467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da277a70d8274edea538084b9a3fade9: Network error: Client connection negotiation failed: client connection to 127.31.210.193:39467: connect: Connection refused (error 111)
I20260808 00:36:36.022992 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 28337
I20260808 00:36:36.032578 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 27812
2026-08-08T00:36:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2453 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T00:36:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:36Z Disabled control of system clock
I20260808 00:36:36.067461 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39043
--webserver_interface=127.31.210.254
--webserver_port=0
--builtin_ntp_servers=127.31.210.212:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:39043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:36.150347 28607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:36.150579 28607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:36.150600 28607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:36.151788 28607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:36.151840 28607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:36.151854 28607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:36.151866 28607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:36.151878 28607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:36.153270 28607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:39043
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:36.153545 28607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:36.153857 28607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:36.158866 28618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:36.158845 28621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:36.158850 28617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:36.159117 28607 server_base.cc:1034] running on GCE node
I20260808 00:36:36.159469 28607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:36.159752 28607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:36.160900 28607 hybrid_clock.cc:648] HybridClock initialized: now 1786149396160885 us; error 31 us; skew 500 ppm
I20260808 00:36:36.162207 28607 webserver.cc:460] Webserver started at http://127.31.210.254:45461/ using document root <none> and password file <none>
I20260808 00:36:36.162487 28607 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:36.162535 28607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:36.162735 28607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:36.163931 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "f70874f56b5d4d9298235668edee49a0"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "b4661e172515fefaebdba9e26587540f"
server_key_iv: "4140f5dd001dfb3066c4f857af68f7e9"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.164312 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "f70874f56b5d4d9298235668edee49a0"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "b4661e172515fefaebdba9e26587540f"
server_key_iv: "4140f5dd001dfb3066c4f857af68f7e9"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.165684 28607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:36.166419 28627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.166595 28607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:36.166688 28607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "f70874f56b5d4d9298235668edee49a0"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "b4661e172515fefaebdba9e26587540f"
server_key_iv: "4140f5dd001dfb3066c4f857af68f7e9"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.166759 28607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:36.188016 28607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:36.188352 28607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:36.188465 28607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:36.192317 28607 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:39043
I20260808 00:36:36.192375 28703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:39043 every 8 connection(s)
I20260808 00:36:36.192773 28607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:36.193780 28704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:36.196094 28704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0: Bootstrap starting.
I20260808 00:36:36.196748 28704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:36.197165 28704 log.cc:824] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:36.197932 28704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0: No bootstrap required, opened a new log
I20260808 00:36:36.199569 28704 raft_consensus.cc:348] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70874f56b5d4d9298235668edee49a0" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39043 } }
I20260808 00:36:36.199716 28704 raft_consensus.cc:374] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:36.199739 28704 raft_consensus.cc:729] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70874f56b5d4d9298235668edee49a0, State: Initialized, Role: FOLLOWER
I20260808 00:36:36.199888 28704 consensus_queue.cc:260] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70874f56b5d4d9298235668edee49a0" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39043 } }
I20260808 00:36:36.199975 28704 raft_consensus.cc:388] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:36.200004 28704 raft_consensus.cc:482] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:36.200039 28704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:36.200584 28704 raft_consensus.cc:504] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70874f56b5d4d9298235668edee49a0" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39043 } }
I20260808 00:36:36.200670 28704 leader_election.cc:302] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f70874f56b5d4d9298235668edee49a0; no voters:
I20260808 00:36:36.200819 28704 leader_election.cc:288] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:36.201090 28710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:36.201270 28704 sys_catalog.cc:564] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:36.201272 28710 raft_consensus.cc:686] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [term 1 LEADER]: Becoming Leader. State: Replica: f70874f56b5d4d9298235668edee49a0, State: Running, Role: LEADER
I20260808 00:36:36.201421 28710 consensus_queue.cc:237] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70874f56b5d4d9298235668edee49a0" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39043 } }
I20260808 00:36:36.201946 28711 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f70874f56b5d4d9298235668edee49a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70874f56b5d4d9298235668edee49a0" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39043 } } }
I20260808 00:36:36.202049 28711 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:36.202391 28723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:36.202570 28715 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f70874f56b5d4d9298235668edee49a0. Latest consensus state: current_term: 1 leader_uuid: "f70874f56b5d4d9298235668edee49a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70874f56b5d4d9298235668edee49a0" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39043 } } }
I20260808 00:36:36.202620 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 28607
I20260808 00:36:36.202657 28715 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:36.202729 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:36.202931 28723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:36.202960 32587 external_mini_cluster.cc:1428] Setting key 9e4c343d0f3fd4d0c1f183c84fad7e25
I20260808 00:36:36.204744 28723 catalog_manager.cc:1347] Generated new cluster ID: 92f3c82a8c804077b06ec154c3ef28f1
I20260808 00:36:36.204802 28723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:36.209237 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.203743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.31.210.254:39043)
0808 00:36:36.203966 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.203975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.204713 (+ 738us) server_negotiation.cc:407] Connection header received
0808 00:36:36.204955 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.204961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.205018 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.205109 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:36.205396 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.206305 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.207119 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.207582 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.207723 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.208415 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:36.208419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:36.208419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:36.208692 (+ 273us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:36.208729 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:36.208741 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:36.208810 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.208902 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.209079 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":110,"threads_started":1}
I20260808 00:36:36.219709 28723 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:36.220250 28723 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:36.232287 28723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0: Generated new TSK 0
I20260808 00:36:36.232540 28723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:36.238585 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.193:0
--local_ip_for_outbound_sockets=127.31.210.193
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39043
--builtin_ntp_servers=127.31.210.212:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:36.329641 28740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:36.329857 28740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:36.329880 28740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:36.331032 28740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:36.331084 28740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:36.331108 28740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.193
I20260808 00:36:36.332465 28740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.210.193
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.193
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:36.332770 28740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:36.333101 28740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:36.339277 28747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:36.339326 28746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:36.339469 28740 server_base.cc:1034] running on GCE node
W20260808 00:36:36.339655 28749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:36.339890 28740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:36.340158 28740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:36.341321 28740 hybrid_clock.cc:648] HybridClock initialized: now 1786149396341311 us; error 39 us; skew 500 ppm
I20260808 00:36:36.342787 28740 webserver.cc:460] Webserver started at http://127.31.210.193:37609/ using document root <none> and password file <none>
I20260808 00:36:36.343156 28740 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:36.343236 28740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:36.343394 28740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:36.344601 28740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data/instance:
uuid: "b8c5eaa265854aa3904cc3455d3a50d6"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "beb9a4970416f57ffa092686f464f0bf"
server_key_iv: "8f7f920673ce9f8827295a8d31e6f31b"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.344985 28740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance:
uuid: "b8c5eaa265854aa3904cc3455d3a50d6"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "beb9a4970416f57ffa092686f464f0bf"
server_key_iv: "8f7f920673ce9f8827295a8d31e6f31b"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.346463 28740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:36:36.347262 28758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.347447 28740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:36:36.347555 28740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/wal
uuid: "b8c5eaa265854aa3904cc3455d3a50d6"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "beb9a4970416f57ffa092686f464f0bf"
server_key_iv: "8f7f920673ce9f8827295a8d31e6f31b"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.347630 28740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:36.367961 28740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:36.368342 28740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:36.368589 28740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:36.368860 28740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:36.369238 28740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:36.369290 28740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.369342 28740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:36.369374 28740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.375411 28740 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.193:41643
I20260808 00:36:36.375490 28925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.193:41643 every 8 connection(s)
I20260808 00:36:36.375847 28740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/data/info.pb
I20260808 00:36:36.381486 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.376662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:57887 (local address 127.31.210.254:39043)
0808 00:36:36.376716 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.376718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.377017 (+ 299us) server_negotiation.cc:407] Connection header received
0808 00:36:36.377574 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.377576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.377591 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.377658 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:36.378375 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.378890 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.379945 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.380142 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.380234 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.381183 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:36.381185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:36.381185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:36.381191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:36.381207 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:36.381212 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:36.381278 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.381371 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.381403 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:36:36.381748 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.376702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39043 (local address 127.31.210.193:57887)
0808 00:36:36.376888 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:36.376924 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:36.377464 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:36.377732 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:36.377737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:36.377747 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:36.378276 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.378281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.379007 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.379008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.379829 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.379831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.380257 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.380258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.380392 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.381018 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:36.381019 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:36.381050 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:36.381076 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:36.381280 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:36.381286 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:36.381290 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:36.381374 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:36:36.381493 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260808 00:36:36.382275 28926 heartbeater.cc:347] Connected to a master server at 127.31.210.254:39043
I20260808 00:36:36.382408 28926 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:36.382625 28926 heartbeater.cc:511] Master 127.31.210.254:39043 requested a full tablet report, sending...
I20260808 00:36:36.383155 28650 ts_manager.cc:194] Registered new tserver with Master: b8c5eaa265854aa3904cc3455d3a50d6 (127.31.210.193:41643)
I20260808 00:36:36.383792 28650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.193:57887
I20260808 00:36:36.385886 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 28740
I20260808 00:36:36.386018 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-0/wal/instance
I20260808 00:36:36.386225 32587 external_mini_cluster.cc:1428] Setting key 94938ebd2e3cdf55d0230cacde4eda95
I20260808 00:36:36.387274 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.194:0
--local_ip_for_outbound_sockets=127.31.210.194
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39043
--builtin_ntp_servers=127.31.210.212:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:36.473779 28932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:36.473999 28932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:36.474021 28932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:36.475157 28932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:36.475200 28932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:36.475225 28932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.194
I20260808 00:36:36.476610 28932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.210.194
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.194
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:36.476868 28932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:36.477187 28932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:36.482419 28938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:36.482441 28939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:36.482419 28941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:36.482877 28932 server_base.cc:1034] running on GCE node
I20260808 00:36:36.483093 28932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:36.483314 28932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:36.484455 28932 hybrid_clock.cc:648] HybridClock initialized: now 1786149396484442 us; error 30 us; skew 500 ppm
I20260808 00:36:36.485773 28932 webserver.cc:460] Webserver started at http://127.31.210.194:38729/ using document root <none> and password file <none>
I20260808 00:36:36.486048 28932 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:36.486122 28932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:36.486271 28932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:36.487372 28932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data/instance:
uuid: "b40e75156ceb41c8a94696fc5540cdab"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "70039825d8ebfc35a301c7e346410251"
server_key_iv: "d17cf6f8d6421bfcb56dc0934a97f678"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.487743 28932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance:
uuid: "b40e75156ceb41c8a94696fc5540cdab"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "70039825d8ebfc35a301c7e346410251"
server_key_iv: "d17cf6f8d6421bfcb56dc0934a97f678"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.489193 28932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:36.489948 28947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.490129 28932 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:36.490235 28932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal
uuid: "b40e75156ceb41c8a94696fc5540cdab"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "70039825d8ebfc35a301c7e346410251"
server_key_iv: "d17cf6f8d6421bfcb56dc0934a97f678"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.490315 28932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:36.498649 28932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:36.498937 28932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:36.499089 28932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:36.499292 28932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:36.499598 28932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:36.499650 28932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.499689 28932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:36.499715 28932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.505897 28932 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.194:33485
I20260808 00:36:36.505949 29112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.194:33485 every 8 connection(s)
I20260808 00:36:36.506281 28932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/data/info.pb
I20260808 00:36:36.511552 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.507072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.194:33195 (local address 127.31.210.254:39043)
0808 00:36:36.507144 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.507147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.507405 (+ 258us) server_negotiation.cc:407] Connection header received
0808 00:36:36.507895 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.507896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.507910 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.507980 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:36.508736 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.509271 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.510218 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.510441 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.510509 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.511282 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:36.511283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:36.511283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:36.511288 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:36.511305 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:36.511310 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:36.511390 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.511463 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.511489 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:36:36.511744 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.507063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39043 (local address 127.31.210.194:33195)
0808 00:36:36.507287 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:36.507320 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:36.507801 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:36.508057 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:36.508060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:36.508069 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:36.508652 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.508656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.509361 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.509361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.510133 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.510135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.510534 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.510535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.510641 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.511147 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:36.511147 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:36.511175 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:36.511198 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:36.511392 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:36.511396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:36.511399 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:36.511451 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:36:36.511540 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":116,"threads_started":1}
I20260808 00:36:36.512277 29113 heartbeater.cc:347] Connected to a master server at 127.31.210.254:39043
I20260808 00:36:36.512383 29113 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:36.512595 29113 heartbeater.cc:511] Master 127.31.210.254:39043 requested a full tablet report, sending...
I20260808 00:36:36.513010 28650 ts_manager.cc:194] Registered new tserver with Master: b40e75156ceb41c8a94696fc5540cdab (127.31.210.194:33485)
I20260808 00:36:36.513334 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 28932
I20260808 00:36:36.513446 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal/instance
I20260808 00:36:36.513540 28650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.194:33195
I20260808 00:36:36.513645 32587 external_mini_cluster.cc:1428] Setting key 5a29b20ff2c1d61f892bedc96c6b287b
I20260808 00:36:36.514550 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.210.195:0
--local_ip_for_outbound_sockets=127.31.210.195
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39043
--builtin_ntp_servers=127.31.210.212:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:36:36.599852 29117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:36.600075 29117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:36.600098 29117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:36.601284 29117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:36.601331 29117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:36.601357 29117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.210.195
I20260808 00:36:36.602744 29117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.210.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.210.195
--webserver_port=0
--tserver_master_addrs=127.31.210.254:39043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.210.195
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:36.602970 29117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:36.603250 29117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:36.608589 29127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:36.608595 29125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:36.608613 29117 server_base.cc:1034] running on GCE node
W20260808 00:36:36.608595 29134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:36.609172 29117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:36.609391 29117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:36.610538 29117 hybrid_clock.cc:648] HybridClock initialized: now 1786149396610528 us; error 36 us; skew 500 ppm
I20260808 00:36:36.611891 29117 webserver.cc:460] Webserver started at http://127.31.210.195:45405/ using document root <none> and password file <none>
I20260808 00:36:36.612154 29117 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:36.612205 29117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:36.612380 29117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:36.613557 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data/instance:
uuid: "c306ac7440b6487c9f8bcea409d16f7d"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "df436b49463545001276bc45e9d14f3e"
server_key_iv: "d9e98cdfb6f9ff489ea3e7180467cfea"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.614003 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance:
uuid: "c306ac7440b6487c9f8bcea409d16f7d"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "df436b49463545001276bc45e9d14f3e"
server_key_iv: "d9e98cdfb6f9ff489ea3e7180467cfea"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.615442 29117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:36:36.616304 29140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.616577 29117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:36.616679 29117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal
uuid: "c306ac7440b6487c9f8bcea409d16f7d"
format_stamp: "Formatted at 2026-08-08 00:36:36 on dist-test-slave-44d2"
server_key: "df436b49463545001276bc45e9d14f3e"
server_key_iv: "d9e98cdfb6f9ff489ea3e7180467cfea"
server_key_version: "encryptionkey@0"
I20260808 00:36:36.616760 29117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:36.659665 29117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:36.660022 29117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:36.660190 29117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:36.660576 29117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:36:36.660959 29117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:36:36.660993 29117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.661016 29117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:36:36.661029 29117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:36.666934 29117 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.195:32775
I20260808 00:36:36.667021 29312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.195:32775 every 8 connection(s)
I20260808 00:36:36.667349 29117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/data/info.pb
I20260808 00:36:36.670920 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 29117
I20260808 00:36:36.671079 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal/instance
I20260808 00:36:36.671271 32587 external_mini_cluster.cc:1428] Setting key f56941636c1f6f2a385c966fc3fb6514
I20260808 00:36:36.673578 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.668162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.195:46753 (local address 127.31.210.254:39043)
0808 00:36:36.668242 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.668244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.668437 (+ 193us) server_negotiation.cc:407] Connection header received
0808 00:36:36.669102 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.669109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.669120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.669190 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:36.670046 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.670634 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.672007 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.672280 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.672333 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.673333 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:36.673335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:36.673335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:36.673339 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:36.673355 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:36.673359 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:36.673398 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.673481 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.673505 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:36:36.673836 29317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.668104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39043 (local address 127.31.210.195:46753)
0808 00:36:36.668303 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:36.668340 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:36.668998 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:36.669312 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:36.669316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:36.669325 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:36:36.669953 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.669965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.670737 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.670738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.671795 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.671798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.672352 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.672353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.672545 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.673180 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:36:36.673181 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:36:36.673210 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:36:36.673239 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:36:36.673414 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:36:36.673419 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:36:36.673421 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:36:36.673482 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:36:36.673566 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260808 00:36:36.674424 29313 heartbeater.cc:347] Connected to a master server at 127.31.210.254:39043
I20260808 00:36:36.674510 29313 heartbeater.cc:464] Registering TS with master...
I20260808 00:36:36.674666 29313 heartbeater.cc:511] Master 127.31.210.254:39043 requested a full tablet report, sending...
I20260808 00:36:36.675035 28650 ts_manager.cc:194] Registered new tserver with Master: c306ac7440b6487c9f8bcea409d16f7d (127.31.210.195:32775)
I20260808 00:36:36.675364 28650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.210.195:46753
I20260808 00:36:36.683097 32587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:36:36.687840 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.684613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49258 (local address 127.31.210.254:39043)
0808 00:36:36.684673 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.684675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.684820 (+ 145us) server_negotiation.cc:407] Connection header received
0808 00:36:36.684914 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.684915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.684930 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.684982 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:36.685186 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.685696 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.686541 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.686757 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.686832 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.687565 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:36.687567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:36.687567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:36.687573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:36.687589 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:36.687593 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:36.687655 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.687719 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.687746 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:36:36.689658 28650 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:36:36.691298 28650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:36:36.698621 29326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.693797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:41643 (local address 127.0.0.1:51492)
0808 00:36:36.693981 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:36.693990 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:36.694028 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:36.694471 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:36.694478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:36.694481 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:36.694575 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.694577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.695943 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.695944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.697426 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.697429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.698292 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.698293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.698375 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.698391 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:36:36.698470 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:36:36.698536 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":110,"thread_start_us":95,"threads_started":1}
I20260808 00:36:36.698904 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.693571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:33485 (local address 127.0.0.1:49490)
0808 00:36:36.693940 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:36.693951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:36.694049 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:36.694470 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:36.694475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:36.694480 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:36.694588 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.694591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.696985 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.696987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.697820 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.697823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.698665 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.698666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.698736 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.698739 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:36:36.698792 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:36:36.698837 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":34,"thread_start_us":83,"threads_started":1}
I20260808 00:36:36.699009 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.694038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.31.210.193:41643)
0808 00:36:36.694278 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.694283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.694295 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:36:36.694338 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.694341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.694357 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.694672 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:36.694735 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.695863 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.697536 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.698212 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.698273 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.698341 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.698853 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.698898 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":122,"threads_started":1}
I20260808 00:36:36.699201 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.693654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49490 (local address 127.31.210.194:33485)
0808 00:36:36.693953 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.693959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.694032 (+ 73us) server_negotiation.cc:407] Connection header received
0808 00:36:36.694140 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.694143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.694191 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.694379 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:36.695969 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.696903 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.697905 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.698458 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.698541 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.698627 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.699065 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.699117 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
I20260808 00:36:36.699636 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.693561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:32775 (local address 127.0.0.1:37332)
0808 00:36:36.693766 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:36.693808 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:36.693876 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:36.694470 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:36.694475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:36.694477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:36.694588 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.694591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.695943 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.695944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.697426 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:36.697429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.699384 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.699385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:36.699450 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.699454 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:36.699510 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:36:36.699564 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":102,"thread_start_us":94,"threads_started":1}
I20260808 00:36:36.699716 28830 tablet_service.cc:1467] Processing CreateTablet for tablet 8e203e1fdd104f2e908237654a4d9c9f (DEFAULT_TABLE table=test-table [id=c92c0124226e4b86bf3830d7ff6de5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:36.699890 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:36.693755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37332 (local address 127.31.210.195:32775)
0808 00:36:36.694321 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:36.694326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:36.694335 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:36.694370 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:36.694373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:36.694389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:36.694458 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:36.694672 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.695846 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.698400 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:36.699205 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:36.699289 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:36.699377 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:36.699752 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 00:36:36.699800 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":77,"threads_started":1}
I20260808 00:36:36.699875 29021 tablet_service.cc:1467] Processing CreateTablet for tablet 8e203e1fdd104f2e908237654a4d9c9f (DEFAULT_TABLE table=test-table [id=c92c0124226e4b86bf3830d7ff6de5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:36.700155 28830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e203e1fdd104f2e908237654a4d9c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:36.700224 29021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e203e1fdd104f2e908237654a4d9c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:36.701222 29206 tablet_service.cc:1467] Processing CreateTablet for tablet 8e203e1fdd104f2e908237654a4d9c9f (DEFAULT_TABLE table=test-table [id=c92c0124226e4b86bf3830d7ff6de5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:36:36.701570 29206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e203e1fdd104f2e908237654a4d9c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:36.703503 29335 tablet_bootstrap.cc:492] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6: Bootstrap starting.
I20260808 00:36:36.703974 29336 tablet_bootstrap.cc:492] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Bootstrap starting.
I20260808 00:36:36.704244 29335 tablet_bootstrap.cc:654] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:36.704639 29335 log.cc:824] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:36.704674 29336 tablet_bootstrap.cc:654] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:36.704988 29336 log.cc:824] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:36.705313 29340 tablet_bootstrap.cc:492] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Bootstrap starting.
I20260808 00:36:36.705533 29335 tablet_bootstrap.cc:492] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6: No bootstrap required, opened a new log
I20260808 00:36:36.705591 29335 ts_tablet_manager.cc:1397] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:36:36.705891 29340 tablet_bootstrap.cc:654] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:36.706175 29340 log.cc:824] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:36.706833 29340 tablet_bootstrap.cc:492] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: No bootstrap required, opened a new log
I20260808 00:36:36.706912 29340 ts_tablet_manager.cc:1397] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:36:36.707298 29335 raft_consensus.cc:348] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:36.707428 29335 raft_consensus.cc:374] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:36.707451 29335 raft_consensus.cc:729] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c5eaa265854aa3904cc3455d3a50d6, State: Initialized, Role: FOLLOWER
I20260808 00:36:36.707571 29335 consensus_queue.cc:260] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:36.707840 29335 ts_tablet_manager.cc:1428] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:36:36.707902 28926 heartbeater.cc:503] Master 127.31.210.254:39043 was elected leader, sending a full tablet report...
I20260808 00:36:36.708417 29336 tablet_bootstrap.cc:492] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: No bootstrap required, opened a new log
I20260808 00:36:36.708516 29336 ts_tablet_manager.cc:1397] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 00:36:36.709429 29340 raft_consensus.cc:348] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:36.709571 29340 raft_consensus.cc:374] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:36.709619 29340 raft_consensus.cc:729] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c306ac7440b6487c9f8bcea409d16f7d, State: Initialized, Role: FOLLOWER
I20260808 00:36:36.709734 29340 consensus_queue.cc:260] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:36.710017 29313 heartbeater.cc:503] Master 127.31.210.254:39043 was elected leader, sending a full tablet report...
I20260808 00:36:36.710238 29340 ts_tablet_manager.cc:1428] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:36:36.710863 29336 raft_consensus.cc:348] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:36.711048 29336 raft_consensus.cc:374] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:36.711100 29336 raft_consensus.cc:729] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40e75156ceb41c8a94696fc5540cdab, State: Initialized, Role: FOLLOWER
I20260808 00:36:36.711216 29336 consensus_queue.cc:260] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:36.711442 29336 ts_tablet_manager.cc:1428] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:36:36.711508 29113 heartbeater.cc:503] Master 127.31.210.254:39043 was elected leader, sending a full tablet report...
W20260808 00:36:36.756906 29114 tablet.cc:2369] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:36.876619 28928 tablet.cc:2369] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:36:36.917958 29314 tablet.cc:2369] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:36:37.075798 29343 raft_consensus.cc:482] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:36:37.075944 29343 raft_consensus.cc:504] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:37.076411 29343 leader_election.cc:288] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c306ac7440b6487c9f8bcea409d16f7d (127.31.210.195:32775), b40e75156ceb41c8a94696fc5540cdab (127.31.210.194:33485)
I20260808 00:36:37.079360 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:37.076708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:51859 (local address 127.31.210.194:33485)
0808 00:36:37.076756 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:37.076759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:37.076873 (+ 114us) server_negotiation.cc:407] Connection header received
0808 00:36:37.076953 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:37.076954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:37.076969 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:37.077032 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:37.077189 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.077661 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.078672 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.079018 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.079100 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:37.079108 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:37.079252 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 00:36:37.079288 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:36:37.079380 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:37.076638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.194:33485 (local address 127.31.210.193:51859)
0808 00:36:37.076796 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:37.076805 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:37.076884 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:37.077039 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:37.077040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:37.077042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:37.077118 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:37.077120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.077745 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.077746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:37.078577 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:37.078580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.079099 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.079100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:37.079180 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:37.079184 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:37.079246 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:36:37.079288 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":14,"spinlock_wait_cycles":13184,"thread_start_us":98,"threads_started":1}
I20260808 00:36:37.079468 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:37.076734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.210.193:45199 (local address 127.31.210.195:32775)
0808 00:36:37.076780 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:37.076782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:37.076857 (+ 75us) server_negotiation.cc:407] Connection header received
0808 00:36:37.076936 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:37.076937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:37.076947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:37.077016 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:36:37.077196 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.077663 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.078706 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.079056 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.079156 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:37.079165 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:37.079353 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 00:36:37.079394 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:36:37.079486 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:37.076632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.195:32775 (local address 127.31.210.193:45199)
0808 00:36:37.076779 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:37.076790 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:36:37.076875 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:36:37.077023 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:36:37.077023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:36:37.077025 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:36:37.077126 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:37.077127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.077743 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.077744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:37.078580 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:36:37.078582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.079202 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.079204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:36:37.079276 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:37.079280 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:36:37.079346 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:36:37.079383 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":32,"thread_start_us":77,"threads_started":1}
I20260808 00:36:37.079856 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e203e1fdd104f2e908237654a4d9c9f" candidate_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c306ac7440b6487c9f8bcea409d16f7d" is_pre_election: true
I20260808 00:36:37.079874 29046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e203e1fdd104f2e908237654a4d9c9f" candidate_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e75156ceb41c8a94696fc5540cdab" is_pre_election: true
I20260808 00:36:37.080019 29046 raft_consensus.cc:2454] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c5eaa265854aa3904cc3455d3a50d6 in term 0.
I20260808 00:36:37.080020 29238 raft_consensus.cc:2454] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c5eaa265854aa3904cc3455d3a50d6 in term 0.
I20260808 00:36:37.080266 28762 leader_election.cc:302] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b40e75156ceb41c8a94696fc5540cdab, b8c5eaa265854aa3904cc3455d3a50d6; no voters:
I20260808 00:36:37.080386 29343 raft_consensus.cc:2781] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:36:37.080448 29343 raft_consensus.cc:482] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:36:37.080499 29343 raft_consensus.cc:3037] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:37.081281 29343 raft_consensus.cc:504] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:37.081425 29343 leader_election.cc:288] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [CANDIDATE]: Term 1 election: Requested vote from peers c306ac7440b6487c9f8bcea409d16f7d (127.31.210.195:32775), b40e75156ceb41c8a94696fc5540cdab (127.31.210.194:33485)
I20260808 00:36:37.081619 29046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e203e1fdd104f2e908237654a4d9c9f" candidate_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e75156ceb41c8a94696fc5540cdab"
I20260808 00:36:37.081619 29238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e203e1fdd104f2e908237654a4d9c9f" candidate_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c306ac7440b6487c9f8bcea409d16f7d"
I20260808 00:36:37.081702 29238 raft_consensus.cc:3037] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:37.081702 29046 raft_consensus.cc:3037] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:37.082597 29238 raft_consensus.cc:2454] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c5eaa265854aa3904cc3455d3a50d6 in term 1.
I20260808 00:36:37.082597 29046 raft_consensus.cc:2454] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c5eaa265854aa3904cc3455d3a50d6 in term 1.
I20260808 00:36:37.082854 28761 leader_election.cc:302] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8c5eaa265854aa3904cc3455d3a50d6, c306ac7440b6487c9f8bcea409d16f7d; no voters:
I20260808 00:36:37.082947 29343 raft_consensus.cc:2781] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:37.083185 29343 raft_consensus.cc:686] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [term 1 LEADER]: Becoming Leader. State: Replica: b8c5eaa265854aa3904cc3455d3a50d6, State: Running, Role: LEADER
I20260808 00:36:37.083317 29343 consensus_queue.cc:237] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } }
I20260808 00:36:37.084403 28648 catalog_manager.cc:5495] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c5eaa265854aa3904cc3455d3a50d6 (127.31.210.193). New cstate: current_term: 1 leader_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8c5eaa265854aa3904cc3455d3a50d6" member_type: VOTER last_known_addr { host: "127.31.210.193" port: 41643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 } health_report { overall_health: UNKNOWN } } }
I20260808 00:36:37.137471 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:37.135011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51496 (local address 127.31.210.193:41643)
0808 00:36:37.135086 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:37.135088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:37.135097 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:37.135144 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:37.135145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:37.135167 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:37.135242 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:36:37.135401 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.135873 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.136674 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:37.136874 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:37.136944 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:37.137128 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:36:37.137239 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:36:37.137333 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:37.137352 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 00:36:37.137375 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:36:37.139430 29238 raft_consensus.cc:1261] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 1 FOLLOWER]: Refusing update from remote peer b8c5eaa265854aa3904cc3455d3a50d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:37.139430 29046 raft_consensus.cc:1261] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 1 FOLLOWER]: Refusing update from remote peer b8c5eaa265854aa3904cc3455d3a50d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:36:37.139814 29343 consensus_queue.cc:1035] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c306ac7440b6487c9f8bcea409d16f7d" member_type: VOTER last_known_addr { host: "127.31.210.195" port: 32775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:37.139976 29348 consensus_queue.cc:1035] T 8e203e1fdd104f2e908237654a4d9c9f P b8c5eaa265854aa3904cc3455d3a50d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b40e75156ceb41c8a94696fc5540cdab" member_type: VOTER last_known_addr { host: "127.31.210.194" port: 33485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:36:37.144763 28650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:37.144879 28650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:36:37.145637 28650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f70874f56b5d4d9298235668edee49a0: Sending DeleteTablet for 3 replicas of tablet 8e203e1fdd104f2e908237654a4d9c9f
I20260808 00:36:37.145983 28830 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e203e1fdd104f2e908237654a4d9c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:37 UTC) from {username='slave'} at 127.0.0.1:51492
I20260808 00:36:37.145995 29206 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e203e1fdd104f2e908237654a4d9c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:37 UTC) from {username='slave'} at 127.0.0.1:37332
I20260808 00:36:37.145995 29021 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e203e1fdd104f2e908237654a4d9c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:36:37 UTC) from {username='slave'} at 127.0.0.1:49490
I20260808 00:36:37.146235 29376 tablet_replica.cc:330] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: stopping tablet replica
I20260808 00:36:37.146346 29376 raft_consensus.cc:2229] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:37.146369 29375 tablet_replica.cc:330] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: stopping tablet replica
I20260808 00:36:37.146464 29376 raft_consensus.cc:2258] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:37.146546 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 28740
I20260808 00:36:37.146653 29375 raft_consensus.cc:2229] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:36:37.146799 29375 raft_consensus.cc:2258] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:36:37.146941 29376 ts_tablet_manager.cc:1905] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:37.147468 29375 ts_tablet_manager.cc:1905] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:36:37.148741 29376 ts_tablet_manager.cc:1918] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:37.148841 29376 log.cc:1196] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-1/wal/wals/8e203e1fdd104f2e908237654a4d9c9f
I20260808 00:36:37.149209 29376 ts_tablet_manager.cc:1939] T 8e203e1fdd104f2e908237654a4d9c9f P b40e75156ceb41c8a94696fc5540cdab: Deleting consensus metadata
I20260808 00:36:37.149926 29375 ts_tablet_manager.cc:1918] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:36:37.150000 29375 log.cc:1196] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Deleting WAL directory at /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149138237870-32587-0/minicluster-data/ts-2/wal/wals/8e203e1fdd104f2e908237654a4d9c9f
I20260808 00:36:37.150203 29375 ts_tablet_manager.cc:1939] T 8e203e1fdd104f2e908237654a4d9c9f P c306ac7440b6487c9f8bcea409d16f7d: Deleting consensus metadata
I20260808 00:36:37.150730 28631 catalog_manager.cc:4841] TS c306ac7440b6487c9f8bcea409d16f7d (127.31.210.195:32775): tablet 8e203e1fdd104f2e908237654a4d9c9f (table test-table [id=c92c0124226e4b86bf3830d7ff6de5b7]) successfully deleted
I20260808 00:36:37.154150 28632 catalog_manager.cc:4841] TS b40e75156ceb41c8a94696fc5540cdab (127.31.210.194:33485): tablet 8e203e1fdd104f2e908237654a4d9c9f (table test-table [id=c92c0124226e4b86bf3830d7ff6de5b7]) successfully deleted
W20260808 00:36:37.158330 28631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.210.193:41643 (error 108)
I20260808 00:36:37.158600 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 28932
I20260808 00:36:37.159077 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:37.158844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.193:41643 (local address 127.0.0.1:51500)
0808 00:36:37.158989 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 00:36:37.159036 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.193:41643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260808 00:36:37.159196 28631 catalog_manager.cc:4584] TS b8c5eaa265854aa3904cc3455d3a50d6 (127.31.210.193:41643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e203e1fdd104f2e908237654a4d9c9f: Network error: Client connection negotiation failed: client connection to 127.31.210.193:41643: connect: Connection refused (error 111)
I20260808 00:36:37.169570 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 29117
I20260808 00:36:37.179659 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 28607
2026-08-08T00:36:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1142 ms)
[----------] 6 tests from EncryptionPolicyTest (10509 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:37 dist-test-slave-44d2 krb5kdc[29381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:37 dist-test-slave-44d2 krb5kdc[29381](info): set up 2 sockets
Aug 08 00:36:37 dist-test-slave-44d2 krb5kdc[29381](info): commencing operation
krb5kdc: starting...
W20260808 00:36:38.425367 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:38 dist-test-slave-44d2 krb5kdc[29381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:38.557989 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:38.643756 29400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:38.643970 29400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:38.643993 29400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:38.645126 29400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:38.645165 29400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:38.645179 29400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:38.645191 29400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:38.645203 29400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:38.647591 29400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39255
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:38.647890 29400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:38.648180 29400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:38.654400 29409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:38.654413 29410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:38.654534 29413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:38.654713 29400 server_base.cc:1034] running on GCE node
I20260808 00:36:38.655055 29400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:38.655368 29400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:38.656553 29400 hybrid_clock.cc:648] HybridClock initialized: now 1786149398656535 us; error 30 us; skew 500 ppm
Aug 08 00:36:38 dist-test-slave-44d2 krb5kdc[29381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149398, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:38.659034 29400 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:38.659538 29400 webserver.cc:460] Webserver started at http://127.0.0.1:45579/ using document root <none> and password file <none>
I20260808 00:36:38.659781 29400 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:38.659848 29400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:38.659993 29400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:38.661136 29400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "3fabc745cd2d4ff89eb4ec29124042f3"
format_stamp: "Formatted at 2026-08-08 00:36:38 on dist-test-slave-44d2"
server_key: "d3962229e9d5b999e5743e22dfab46c7"
server_key_iv: "03f343744aa7efcca713f4ee0ecb5cf7"
server_key_version: "encryptionkey@0"
I20260808 00:36:38.661499 29400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "3fabc745cd2d4ff89eb4ec29124042f3"
format_stamp: "Formatted at 2026-08-08 00:36:38 on dist-test-slave-44d2"
server_key: "d3962229e9d5b999e5743e22dfab46c7"
server_key_iv: "03f343744aa7efcca713f4ee0ecb5cf7"
server_key_version: "encryptionkey@0"
I20260808 00:36:38.662822 29400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:36:38.663537 29421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:38.663703 29400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:38.663800 29400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "3fabc745cd2d4ff89eb4ec29124042f3"
format_stamp: "Formatted at 2026-08-08 00:36:38 on dist-test-slave-44d2"
server_key: "d3962229e9d5b999e5743e22dfab46c7"
server_key_iv: "03f343744aa7efcca713f4ee0ecb5cf7"
server_key_version: "encryptionkey@0"
I20260808 00:36:38.663877 29400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:38.684588 29400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:38.686798 29400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:38.686966 29400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:38.691067 29400 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39255
I20260808 00:36:38.691105 29494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39255 every 8 connection(s)
I20260808 00:36:38.691505 29400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:38.692365 29495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:38.694545 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 29400
I20260808 00:36:38.694682 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:38.694880 29495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3: Bootstrap starting.
I20260808 00:36:38.694916 32587 external_mini_cluster.cc:1428] Setting key f9bc0803c3ff93b3cf5e1408f5816ced
I20260808 00:36:38.695647 29495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:38.695945 29495 log.cc:824] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:38.696744 29495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3: No bootstrap required, opened a new log
I20260808 00:36:38.698485 29495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39255 } }
I20260808 00:36:38.698647 29495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:38.698693 29495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fabc745cd2d4ff89eb4ec29124042f3, State: Initialized, Role: FOLLOWER
I20260808 00:36:38.698822 29495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39255 } }
I20260808 00:36:38.698900 29495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:38.698944 29495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:38.698992 29495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:38.699659 29495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39255 } }
I20260808 00:36:38.699774 29495 leader_election.cc:302] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fabc745cd2d4ff89eb4ec29124042f3; no voters:
I20260808 00:36:38.699937 29495 leader_election.cc:288] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:38.699982 29500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:38.700178 29500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [term 1 LEADER]: Becoming Leader. State: Replica: 3fabc745cd2d4ff89eb4ec29124042f3, State: Running, Role: LEADER
I20260808 00:36:38.700280 29495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:38.700335 29500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39255 } }
I20260808 00:36:38.700791 29503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fabc745cd2d4ff89eb4ec29124042f3. Latest consensus state: current_term: 1 leader_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39255 } } }
I20260808 00:36:38.700978 29503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:38.701442 29512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:38.702028 29512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:36:38 dist-test-slave-44d2 krb5kdc[29381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:36:38.702016 29504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fabc745cd2d4ff89eb4ec29124042f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39255 } } }
I20260808 00:36:38.702937 29504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:38.703810 29512 catalog_manager.cc:1347] Generated new cluster ID: a77613b458dd4cec8a1dfc54181fa9da
I20260808 00:36:38.703864 29512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:38.706905 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:38.695451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55375 (local address 127.0.0.1:39255)
0808 00:36:38.695669 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:38.695680 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:38.695726 (+ 46us) server_negotiation.cc:407] Connection header received
0808 00:36:38.696321 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:38.696331 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:38.696397 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:38.696604 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:38.697312 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:38.698193 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:38.699117 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:38.699735 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:38.699895 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:38.702892 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:38.702897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:38.702897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:38.702910 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:38.705187 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:38.705624 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:38.705627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:38.705628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:38.705749 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:38.705983 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:38.705984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:38.705985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:38.706308 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:38.706483 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:38.706580 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 00:36:38.706688 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260808 00:36:38.708002 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:38.717617 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:38.709675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49983 (local address 127.0.0.1:39255)
0808 00:36:38.709715 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:38.709717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:38.709920 (+ 203us) server_negotiation.cc:407] Connection header received
0808 00:36:38.709958 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:38.709959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:38.709971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:38.710056 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:38.710713 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:38.711623 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:38.712749 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:38.713245 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:38.713407 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:38.714762 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:38.714764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:38.714764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:38.714771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:38.716360 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:38.716804 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:38.716805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:38.716806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:38.716863 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:38.717110 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:38.717110 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:38.717111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:38.717231 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:38.717318 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:38.717468 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 00:36:38.717530 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:36:38.724881 29512 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:38.725620 29512 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:38.730958 29512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fabc745cd2d4ff89eb4ec29124042f3: Generated new TSK 0
I20260808 00:36:38.731151 29512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:38.740250 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 29400
W20260808 00:36:38.758952 29379 connection.cc:535] client connection to 127.0.0.1:39255 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39255): Connection reset by peer (error 104)
2026-08-08T00:36:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:38 dist-test-slave-44d2 krb5kdc[29533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:38 dist-test-slave-44d2 krb5kdc[29533](info): set up 2 sockets
Aug 08 00:36:38 dist-test-slave-44d2 krb5kdc[29533](info): commencing operation
krb5kdc: starting...
W20260808 00:36:40.061129 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:40 dist-test-slave-44d2 krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:40.199292 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:40.285148 29553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:40.285403 29553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:40.285444 29553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:40.286630 29553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:40.286711 29553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:40.286736 29553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:40.286758 29553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:40.286784 29553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:40.288403 29553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40379
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:40.288733 29553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:40.289036 29553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:40.294339 29563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:40.294359 29562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:40.294298 29565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:40.294559 29553 server_base.cc:1034] running on GCE node
I20260808 00:36:40.294806 29553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:40.295122 29553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:40.296290 29553 hybrid_clock.cc:648] HybridClock initialized: now 1786149400296265 us; error 49 us; skew 500 ppm
Aug 08 00:36:40 dist-test-slave-44d2 krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:40.299185 29553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:40.299774 29553 webserver.cc:460] Webserver started at http://127.0.0.1:41391/ using document root <none> and password file <none>
I20260808 00:36:40.300091 29553 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:40.300165 29553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:40.300338 29553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:40.301628 29553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "07b534e0262e4d6599986399354d2574"
format_stamp: "Formatted at 2026-08-08 00:36:40 on dist-test-slave-44d2"
server_key: "d04f4dcac967595a34625cd134694f9e"
server_key_iv: "20c3c24b4fdee6e525437f774f92f42e"
server_key_version: "encryptionkey@0"
I20260808 00:36:40.302253 29553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "07b534e0262e4d6599986399354d2574"
format_stamp: "Formatted at 2026-08-08 00:36:40 on dist-test-slave-44d2"
server_key: "d04f4dcac967595a34625cd134694f9e"
server_key_iv: "20c3c24b4fdee6e525437f774f92f42e"
server_key_version: "encryptionkey@0"
I20260808 00:36:40.304030 29553 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:36:40.304934 29575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:40.305110 29553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:40.305226 29553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "07b534e0262e4d6599986399354d2574"
format_stamp: "Formatted at 2026-08-08 00:36:40 on dist-test-slave-44d2"
server_key: "d04f4dcac967595a34625cd134694f9e"
server_key_iv: "20c3c24b4fdee6e525437f774f92f42e"
server_key_version: "encryptionkey@0"
I20260808 00:36:40.305308 29553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:40.321009 29553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:40.323974 29553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:40.324172 29553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:40.329016 29553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40379
I20260808 00:36:40.329066 29652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40379 every 8 connection(s)
I20260808 00:36:40.329471 29553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:40.330399 29653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:40.332780 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574: Bootstrap starting.
I20260808 00:36:40.333387 29653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:40.333688 29653 log.cc:824] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:40.334404 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574: No bootstrap required, opened a new log
I20260808 00:36:40.335235 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 29553
I20260808 00:36:40.335359 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:40.335563 32587 external_mini_cluster.cc:1428] Setting key fa6567e0e34d73701e4876fb1e4365b4
I20260808 00:36:40.336598 29653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b534e0262e4d6599986399354d2574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260808 00:36:40.336759 29653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:40.336822 29653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b534e0262e4d6599986399354d2574, State: Initialized, Role: FOLLOWER
I20260808 00:36:40.337016 29653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b534e0262e4d6599986399354d2574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260808 00:36:40.337117 29653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:40.337160 29653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:40.337208 29653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:40.337998 29653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b534e0262e4d6599986399354d2574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260808 00:36:40.338112 29653 leader_election.cc:302] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07b534e0262e4d6599986399354d2574; no voters:
I20260808 00:36:40.338264 29653 leader_election.cc:288] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:40.338318 29660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:40.338521 29653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:40.338521 29660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [term 1 LEADER]: Becoming Leader. State: Replica: 07b534e0262e4d6599986399354d2574, State: Running, Role: LEADER
I20260808 00:36:40.338723 29660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b534e0262e4d6599986399354d2574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260808 00:36:40.339846 29663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07b534e0262e4d6599986399354d2574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b534e0262e4d6599986399354d2574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } } }
I20260808 00:36:40.339954 29663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:40.340179 29664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07b534e0262e4d6599986399354d2574. Latest consensus state: current_term: 1 leader_uuid: "07b534e0262e4d6599986399354d2574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b534e0262e4d6599986399354d2574" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } } }
I20260808 00:36:40.340409 29664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574 [sys.catalog]: This master's current role is: LEADER
W20260808 00:36:40.340725 29682 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:36:40.340791 29682 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:36:40.340885 29683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:40.341073 29683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:40.342532 29658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:40.336133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44421 (local address 127.0.0.1:40379)
0808 00:36:40.336329 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:40.336342 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:40.336372 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:36:40.336527 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:40.336535 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:40.336599 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:40.336742 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:40.337032 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:40.338877 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:40.340071 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:40.340780 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:40.341239 (+ 459us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:40.341712 (+ 473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:40.341716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:40.341716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:40.341925 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:40.341965 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:40.341979 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:40.342220 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:40.342264 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 00:36:40.342293 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260808 00:36:40.342588 29683 catalog_manager.cc:1347] Generated new cluster ID: ae9856034530408a96be1c1cf233540c
I20260808 00:36:40.342754 29683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:40.343648 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:40.348670 29658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:40.345154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56317 (local address 127.0.0.1:40379)
0808 00:36:40.345189 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:40.345190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:40.345245 (+ 55us) server_negotiation.cc:407] Connection header received
0808 00:36:40.345306 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:40.345306 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:40.345316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:40.345385 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:40.345528 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:40.346257 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:40.347268 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:40.347613 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:40.347683 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:40.348385 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:40.348386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:40.348387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:40.348391 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:40.348407 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:40.348412 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:40.348504 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:40.348582 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 00:36:40.348596 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:36:40.361691 29683 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:40.362252 29683 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:40.369041 29683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07b534e0262e4d6599986399354d2574: Generated new TSK 0
I20260808 00:36:40.369262 29683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:40.372351 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 29553
W20260808 00:36:40.390419 29531 connection.cc:535] client connection to 127.0.0.1:40379 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40379): Connection reset by peer (error 104)
2026-08-08T00:36:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1631 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:40 dist-test-slave-44d2 krb5kdc[29699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:40 dist-test-slave-44d2 krb5kdc[29699](info): set up 2 sockets
Aug 08 00:36:40 dist-test-slave-44d2 krb5kdc[29699](info): commencing operation
krb5kdc: starting...
W20260808 00:36:41.696671 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:41 dist-test-slave-44d2 krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:41.835366 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:41.923736 29716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:41.923988 29716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:41.924016 29716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:41.925437 29716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:41.925498 29716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:41.925515 29716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:41.925530 29716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:41.925544 29716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:41.927260 29716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40353
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:41.927557 29716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:41.927831 29716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:41.933682 29728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:41.933706 29726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:41.933706 29724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:41.933988 29716 server_base.cc:1034] running on GCE node
I20260808 00:36:41.934295 29716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:41.934588 29716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:41.935791 29716 hybrid_clock.cc:648] HybridClock initialized: now 1786149401935769 us; error 41 us; skew 500 ppm
Aug 08 00:36:41 dist-test-slave-44d2 krb5kdc[29699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149401, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:41.938767 29716 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:41.939313 29716 webserver.cc:460] Webserver started at http://127.0.0.1:42069/ using document root <none> and password file <none>
I20260808 00:36:41.939553 29716 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:41.939597 29716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:41.939692 29716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:41.940855 29716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "539aaf6731a343da9e77f38214cfa5ee"
format_stamp: "Formatted at 2026-08-08 00:36:41 on dist-test-slave-44d2"
server_key: "49500b797f60115c4d8b1042e20bf3f9"
server_key_iv: "92c792409d4ac64a01565556660cd2ee"
server_key_version: "encryptionkey@0"
I20260808 00:36:41.941179 29716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "539aaf6731a343da9e77f38214cfa5ee"
format_stamp: "Formatted at 2026-08-08 00:36:41 on dist-test-slave-44d2"
server_key: "49500b797f60115c4d8b1042e20bf3f9"
server_key_iv: "92c792409d4ac64a01565556660cd2ee"
server_key_version: "encryptionkey@0"
I20260808 00:36:41.942631 29716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:41.943537 29735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:41.943737 29716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:41.943835 29716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "539aaf6731a343da9e77f38214cfa5ee"
format_stamp: "Formatted at 2026-08-08 00:36:41 on dist-test-slave-44d2"
server_key: "49500b797f60115c4d8b1042e20bf3f9"
server_key_iv: "92c792409d4ac64a01565556660cd2ee"
server_key_version: "encryptionkey@0"
I20260808 00:36:41.943894 29716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:41.947296 29716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:41.949563 29716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:41.949700 29716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:41.954149 29716 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40353
I20260808 00:36:41.954177 29808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40353 every 8 connection(s)
I20260808 00:36:41.954576 29716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:41.955677 29810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:41.958093 29810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee: Bootstrap starting.
I20260808 00:36:41.958695 29810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:41.958986 29810 log.cc:824] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:41.959743 29810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee: No bootstrap required, opened a new log
I20260808 00:36:41.961236 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 29716
I20260808 00:36:41.961369 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:41.961582 32587 external_mini_cluster.cc:1428] Setting key 637a2153554a3b7667a13a68c821d9d3
I20260808 00:36:41.961824 29810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539aaf6731a343da9e77f38214cfa5ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40353 } }
I20260808 00:36:41.962004 29810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:41.962109 29810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 539aaf6731a343da9e77f38214cfa5ee, State: Initialized, Role: FOLLOWER
I20260808 00:36:41.962253 29810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539aaf6731a343da9e77f38214cfa5ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40353 } }
I20260808 00:36:41.962349 29810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:41.962396 29810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:41.962442 29810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:41.963586 29810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539aaf6731a343da9e77f38214cfa5ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40353 } }
I20260808 00:36:41.963703 29810 leader_election.cc:302] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 539aaf6731a343da9e77f38214cfa5ee; no voters:
I20260808 00:36:41.963888 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:41.962110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42537 (local address 127.0.0.1:40353)
0808 00:36:41.962301 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:41.962311 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:41.962345 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:36:41.962468 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:41.962476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:41.962507 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:41.962675 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:41.963375 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:41.963379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:41.963380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:41.963385 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:41.963434 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:41.963449 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:41.963529 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:41.963589 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 00:36:41.963634 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260808 00:36:41.963896 29810 leader_election.cc:288] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:41.963953 29819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:41.964100 29819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [term 1 LEADER]: Becoming Leader. State: Replica: 539aaf6731a343da9e77f38214cfa5ee, State: Running, Role: LEADER
I20260808 00:36:41.964192 29810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:41.964190 29819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539aaf6731a343da9e77f38214cfa5ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40353 } }
I20260808 00:36:41.964882 29821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "539aaf6731a343da9e77f38214cfa5ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539aaf6731a343da9e77f38214cfa5ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40353 } } }
I20260808 00:36:41.964932 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:41.964975 29821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:41.965205 29828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:41.965736 29828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:41.966758 29820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 539aaf6731a343da9e77f38214cfa5ee. Latest consensus state: current_term: 1 leader_uuid: "539aaf6731a343da9e77f38214cfa5ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "539aaf6731a343da9e77f38214cfa5ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40353 } } }
I20260808 00:36:41.966845 29820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:41.967423 29828 catalog_manager.cc:1347] Generated new cluster ID: e7d0b8dc18394dfc93c2833576a46e5c
I20260808 00:36:41.967479 29828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:41.968034 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:41.966591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47207 (local address 127.0.0.1:40353)
0808 00:36:41.966779 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:41.966781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:41.967009 (+ 228us) server_negotiation.cc:407] Connection header received
0808 00:36:41.967092 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:41.967093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:41.967107 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:41.967182 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:41.967832 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:41.967833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:41.967833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:41.967837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:41.967850 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:41.967854 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:41.967902 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:41.967953 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:36:41.967968 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260808 00:36:41.988667 29828 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:41.988873 29828 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:41.993698 29828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 539aaf6731a343da9e77f38214cfa5ee: Generated new TSK 0
I20260808 00:36:41.993904 29828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:41.998824 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 29716
2026-08-08T00:36:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:42 dist-test-slave-44d2 krb5kdc[29856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:42 dist-test-slave-44d2 krb5kdc[29856](info): set up 2 sockets
Aug 08 00:36:42 dist-test-slave-44d2 krb5kdc[29856](info): commencing operation
krb5kdc: starting...
W20260808 00:36:43.241972 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:43 dist-test-slave-44d2 krb5kdc[29856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:43.371814 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:43.456282 29880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:43.456557 29880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:43.456600 29880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:43.457760 29880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:43.457834 29880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:43.457857 29880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:43.457885 29880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:43.457918 29880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:43.459383 29880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:43.459676 29880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:43.459913 29880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:43.464927 29885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:43.464916 29888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:43.464916 29886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:43.464959 29880 server_base.cc:1034] running on GCE node
I20260808 00:36:43.465464 29880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:43.465773 29880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:43.466943 29880 hybrid_clock.cc:648] HybridClock initialized: now 1786149403466937 us; error 39 us; skew 500 ppm
Aug 08 00:36:43 dist-test-slave-44d2 krb5kdc[29856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149403, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:43.475904 29880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:43.476387 29880 webserver.cc:460] Webserver started at http://127.0.0.1:34075/ using document root <none> and password file <none>
I20260808 00:36:43.476675 29880 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:43.476728 29880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:43.476828 29880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:43.477980 29880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f"
format_stamp: "Formatted at 2026-08-08 00:36:43 on dist-test-slave-44d2"
server_key: "beaaf2a0e8889f4d1284f1ac49b68506"
server_key_iv: "76f15bf66960abfbc6e504c1b2a9585e"
server_key_version: "encryptionkey@0"
I20260808 00:36:43.478319 29880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f"
format_stamp: "Formatted at 2026-08-08 00:36:43 on dist-test-slave-44d2"
server_key: "beaaf2a0e8889f4d1284f1ac49b68506"
server_key_iv: "76f15bf66960abfbc6e504c1b2a9585e"
server_key_version: "encryptionkey@0"
I20260808 00:36:43.479631 29880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:43.480418 29897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:43.480641 29880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:43.480743 29880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f"
format_stamp: "Formatted at 2026-08-08 00:36:43 on dist-test-slave-44d2"
server_key: "beaaf2a0e8889f4d1284f1ac49b68506"
server_key_iv: "76f15bf66960abfbc6e504c1b2a9585e"
server_key_version: "encryptionkey@0"
I20260808 00:36:43.480824 29880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:43.496872 29880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:43.498960 29880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:43.499122 29880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:43.503298 29880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36029
I20260808 00:36:43.503368 29970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36029 every 8 connection(s)
I20260808 00:36:43.503731 29880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:43.504662 29972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:43.506899 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f: Bootstrap starting.
I20260808 00:36:43.507440 29972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:43.507679 29972 log.cc:824] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:43.508006 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 29880
I20260808 00:36:43.508138 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:43.508363 32587 external_mini_cluster.cc:1428] Setting key 9480d88ac2a2b56738aedb86639caf2c
I20260808 00:36:43.508576 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f: No bootstrap required, opened a new log
I20260808 00:36:43.510586 29972 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36029 } }
I20260808 00:36:43.510742 29972 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:43.510808 29972 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2f2a75c9eac4ac9bb9c11929a25d69f, State: Initialized, Role: FOLLOWER
I20260808 00:36:43.510941 29972 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36029 } }
I20260808 00:36:43.511049 29972 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:43.511092 29972 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:43.511140 29972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:43.511688 29972 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36029 } }
I20260808 00:36:43.511811 29972 leader_election.cc:302] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2f2a75c9eac4ac9bb9c11929a25d69f; no voters:
I20260808 00:36:43.512027 29972 leader_election.cc:288] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:43.512044 29987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:43.512171 29987 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [term 1 LEADER]: Becoming Leader. State: Replica: a2f2a75c9eac4ac9bb9c11929a25d69f, State: Running, Role: LEADER
I20260808 00:36:43.512298 29972 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:43.512315 29987 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36029 } }
I20260808 00:36:43.512873 29987 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2f2a75c9eac4ac9bb9c11929a25d69f. Latest consensus state: current_term: 1 leader_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36029 } } }
I20260808 00:36:43.512980 29988 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f2a75c9eac4ac9bb9c11929a25d69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36029 } } }
I20260808 00:36:43.513121 29988 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:43.513335 29987 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:43.513536 30001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:43.514138 30001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:43.515856 30001 catalog_manager.cc:1347] Generated new cluster ID: ec7ded5f8595487ca51ea716ec0ed469
I20260808 00:36:43.515913 30001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:36:43 dist-test-slave-44d2 krb5kdc[29856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:36:43.520764 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:43.508940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51659 (local address 127.0.0.1:36029)
0808 00:36:43.509125 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:43.509133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:43.509158 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:36:43.509880 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:43.509888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:43.509936 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:43.510138 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:43.510797 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:43.512251 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:43.513332 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:43.514265 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:43.514766 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:43.517310 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:43.517315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:43.517315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:43.517328 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:43.519069 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:43.519467 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:43.519470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:43.519471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:43.519573 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:43.519794 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:43.519795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:43.519796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:43.520052 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:43.520227 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:43.520299 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 00:36:43.520577 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260808 00:36:43.521720 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:43.529418 30001 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:43.530102 30001 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:43.533946 30001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2f2a75c9eac4ac9bb9c11929a25d69f: Generated new TSK 0
I20260808 00:36:43.534169 30001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:43.534579 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:43.524895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38235 (local address 127.0.0.1:36029)
0808 00:36:43.524945 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:43.524947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:43.527961 (+ 3014us) server_negotiation.cc:407] Connection header received
0808 00:36:43.528075 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:43.528076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:43.528089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:43.528161 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:43.528804 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:43.529482 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:43.530238 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:43.530461 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:43.530537 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:43.531787 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:43.531788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:43.531789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:43.531796 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:43.533308 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:43.533750 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:43.533752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:43.533752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:43.533811 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:43.534011 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:43.534012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:43.534013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:43.534136 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:43.534253 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:43.534366 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:36:43.534455 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:36:43.536470 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 29880
2026-08-08T00:36:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:43 dist-test-slave-44d2 krb5kdc[30021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:43 dist-test-slave-44d2 krb5kdc[30021](info): set up 2 sockets
Aug 08 00:36:43 dist-test-slave-44d2 krb5kdc[30021](info): commencing operation
krb5kdc: starting...
W20260808 00:36:44.778694 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:44 dist-test-slave-44d2 krb5kdc[30021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:44.915834 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:45.001034 30038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:45.001294 30038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:45.001334 30038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:45.002516 30038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:45.002597 30038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:45.002619 30038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:45.002640 30038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:45.002671 30038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:45.004174 30038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:45.004467 30038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:45.004739 30038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:45.009790 30043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:45.009799 30049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:45.009812 30047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:45.010140 30038 server_base.cc:1034] running on GCE node
I20260808 00:36:45.010365 30038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:45.010618 30038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:45.011762 30038 hybrid_clock.cc:648] HybridClock initialized: now 1786149405011745 us; error 31 us; skew 500 ppm
Aug 08 00:36:45 dist-test-slave-44d2 krb5kdc[30021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149405, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:45.014442 30038 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:45.014894 30038 webserver.cc:460] Webserver started at http://127.0.0.1:44179/ using document root <none> and password file <none>
I20260808 00:36:45.015156 30038 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:45.015204 30038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:45.015376 30038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:45.016541 30038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "f36621df341f455ab9ac35f6dde1209f"
format_stamp: "Formatted at 2026-08-08 00:36:45 on dist-test-slave-44d2"
server_key: "8d4c40ebad8f03c0d97ca2ba47ba0272"
server_key_iv: "e2eeaad6c5dbebbef472684073f15d3c"
server_key_version: "encryptionkey@0"
I20260808 00:36:45.016924 30038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "f36621df341f455ab9ac35f6dde1209f"
format_stamp: "Formatted at 2026-08-08 00:36:45 on dist-test-slave-44d2"
server_key: "8d4c40ebad8f03c0d97ca2ba47ba0272"
server_key_iv: "e2eeaad6c5dbebbef472684073f15d3c"
server_key_version: "encryptionkey@0"
I20260808 00:36:45.018304 30038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:36:45.019028 30061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:45.019204 30038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:45.019316 30038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "f36621df341f455ab9ac35f6dde1209f"
format_stamp: "Formatted at 2026-08-08 00:36:45 on dist-test-slave-44d2"
server_key: "8d4c40ebad8f03c0d97ca2ba47ba0272"
server_key_iv: "e2eeaad6c5dbebbef472684073f15d3c"
server_key_version: "encryptionkey@0"
I20260808 00:36:45.019392 30038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:45.031315 30038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:45.033433 30038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:45.033607 30038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:45.037711 30038 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40691
I20260808 00:36:45.037766 30129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40691 every 8 connection(s)
I20260808 00:36:45.038151 30038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:45.039080 30131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:45.041540 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 30038
I20260808 00:36:45.041602 30131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f: Bootstrap starting.
I20260808 00:36:45.041674 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:45.041898 32587 external_mini_cluster.cc:1428] Setting key a7666ac187a529eaf35688906d902858
I20260808 00:36:45.042279 30131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:45.042843 30131 log.cc:824] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:45.043589 30131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f: No bootstrap required, opened a new log
I20260808 00:36:45.045810 30131 raft_consensus.cc:348] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36621df341f455ab9ac35f6dde1209f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260808 00:36:45.045970 30131 raft_consensus.cc:374] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:45.045995 30131 raft_consensus.cc:729] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36621df341f455ab9ac35f6dde1209f, State: Initialized, Role: FOLLOWER
I20260808 00:36:45.046118 30131 consensus_queue.cc:260] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36621df341f455ab9ac35f6dde1209f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260808 00:36:45.046193 30131 raft_consensus.cc:388] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:45.046214 30131 raft_consensus.cc:482] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:45.046272 30131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:45.046969 30131 raft_consensus.cc:504] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36621df341f455ab9ac35f6dde1209f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260808 00:36:45.047092 30131 leader_election.cc:302] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f36621df341f455ab9ac35f6dde1209f; no voters:
I20260808 00:36:45.047300 30131 leader_election.cc:288] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:45.047370 30138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:45.047587 30131 sys_catalog.cc:564] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:45.047608 30138 raft_consensus.cc:686] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [term 1 LEADER]: Becoming Leader. State: Replica: f36621df341f455ab9ac35f6dde1209f, State: Running, Role: LEADER
I20260808 00:36:45.047787 30138 consensus_queue.cc:237] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36621df341f455ab9ac35f6dde1209f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260808 00:36:45.048168 30142 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f36621df341f455ab9ac35f6dde1209f. Latest consensus state: current_term: 1 leader_uuid: "f36621df341f455ab9ac35f6dde1209f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36621df341f455ab9ac35f6dde1209f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } } }
I20260808 00:36:45.048177 30139 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f36621df341f455ab9ac35f6dde1209f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36621df341f455ab9ac35f6dde1209f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } } }
I20260808 00:36:45.048341 30139 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:45.048429 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:45.042444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58113 (local address 127.0.0.1:40691)
0808 00:36:45.042687 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:45.042695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:45.042722 (+ 27us) server_negotiation.cc:407] Connection header received
0808 00:36:45.043221 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:45.043228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:45.043276 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:45.043394 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:45.043704 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:45.044646 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:45.045861 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:45.046550 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:45.046652 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:45.047453 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:45.047457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:45.047457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:45.047689 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:45.047730 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:45.047746 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:45.047811 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:45.048042 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 00:36:45.048120 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":95,"threads_started":1}
I20260808 00:36:45.048605 30142 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:45.048733 30151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:45.049252 30151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:45.049381 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:45.051355 30151 catalog_manager.cc:1347] Generated new cluster ID: 6612c9a2ef4547d1959a2069b196eb6f
I20260808 00:36:45.051414 30151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:45.055335 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:45.051169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42635 (local address 127.0.0.1:40691)
0808 00:36:45.051205 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:45.051206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:45.051299 (+ 93us) server_negotiation.cc:407] Connection header received
0808 00:36:45.052662 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:45.052663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:45.052675 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:45.052721 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:45.052887 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:45.053389 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:45.054115 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:45.054328 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:45.054416 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:45.055093 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:45.055095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:45.055095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:45.055099 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:45.055114 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:45.055118 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:45.055170 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:45.055228 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 00:36:45.055252 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":18}
I20260808 00:36:45.082165 30151 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:45.082999 30151 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:45.089975 30151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f36621df341f455ab9ac35f6dde1209f: Generated new TSK 0
I20260808 00:36:45.090322 30151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:45.100092 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 30038
2026-08-08T00:36:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:45 dist-test-slave-44d2 krb5kdc[30176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:45 dist-test-slave-44d2 krb5kdc[30176](info): set up 2 sockets
Aug 08 00:36:45 dist-test-slave-44d2 krb5kdc[30176](info): commencing operation
krb5kdc: starting...
W20260808 00:36:46.346129 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:46 dist-test-slave-44d2 krb5kdc[30176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:46.478935 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:46.564558 30191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:46.564814 30191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:46.564858 30191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:46.566051 30191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:46.566128 30191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:46.566152 30191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:46.566174 30191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:46.566203 30191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:46.567720 30191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:46.568029 30191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:46.568277 30191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:46.573504 30204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:46.573524 30200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:46.573573 30201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:46.573822 30191 server_base.cc:1034] running on GCE node
I20260808 00:36:46.574117 30191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:46.574450 30191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:46.575609 30191 hybrid_clock.cc:648] HybridClock initialized: now 1786149406575589 us; error 38 us; skew 500 ppm
Aug 08 00:36:46 dist-test-slave-44d2 krb5kdc[30176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149406, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:46.578240 30191 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:46.578717 30191 webserver.cc:460] Webserver started at http://127.0.0.1:45067/ using document root <none> and password file <none>
I20260808 00:36:46.578953 30191 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:46.579025 30191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:46.579185 30191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:46.580279 30191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "63e264c3f2384804bf10b53c14b3243e"
format_stamp: "Formatted at 2026-08-08 00:36:46 on dist-test-slave-44d2"
server_key: "f2bbf3f9f1a66087f6e7a378b76f04cd"
server_key_iv: "4383154ed9ee55ef2bd9c31b67db04e5"
server_key_version: "encryptionkey@0"
I20260808 00:36:46.580684 30191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "63e264c3f2384804bf10b53c14b3243e"
format_stamp: "Formatted at 2026-08-08 00:36:46 on dist-test-slave-44d2"
server_key: "f2bbf3f9f1a66087f6e7a378b76f04cd"
server_key_iv: "4383154ed9ee55ef2bd9c31b67db04e5"
server_key_version: "encryptionkey@0"
I20260808 00:36:46.582062 30191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:46.582794 30213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:46.582969 30191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:36:46.583076 30191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "63e264c3f2384804bf10b53c14b3243e"
format_stamp: "Formatted at 2026-08-08 00:36:46 on dist-test-slave-44d2"
server_key: "f2bbf3f9f1a66087f6e7a378b76f04cd"
server_key_iv: "4383154ed9ee55ef2bd9c31b67db04e5"
server_key_version: "encryptionkey@0"
I20260808 00:36:46.583154 30191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:46.586624 30191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:46.589000 30191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:46.589169 30191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:46.593150 30191 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43621
I20260808 00:36:46.593199 30290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43621 every 8 connection(s)
I20260808 00:36:46.593556 30191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:46.594592 30291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:46.594846 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 30191
I20260808 00:36:46.594992 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:46.595194 32587 external_mini_cluster.cc:1428] Setting key d891d9d3db8c4aaddccd89529d452ee7
I20260808 00:36:46.597790 30291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e: Bootstrap starting.
I20260808 00:36:46.597839 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:46.595728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39007 (local address 127.0.0.1:43621)
0808 00:36:46.595966 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:46.595975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:46.596019 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:36:46.596580 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:46.596589 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:46.596626 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:46.596768 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:46.597431 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:46.597435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:46.597435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:46.597440 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:46.597484 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:46.597499 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:46.597576 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:46.597614 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 00:36:46.597655 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
I20260808 00:36:46.598594 30291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:46.598906 30291 log.cc:824] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:46.598964 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:46.599720 30291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e: No bootstrap required, opened a new log
I20260808 00:36:46.601588 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:46.600589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33923 (local address 127.0.0.1:43621)
0808 00:36:46.600629 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:46.600631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:46.600774 (+ 143us) server_negotiation.cc:407] Connection header received
0808 00:36:46.600831 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:46.600831 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:46.600836 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:46.600882 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:46.601419 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:46.601420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:46.601420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:46.601424 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:46.601436 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:46.601440 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:46.601478 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:46.601531 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 00:36:46.601541 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:36:46.602443 30291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e264c3f2384804bf10b53c14b3243e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43621 } }
I20260808 00:36:46.602588 30291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:46.602612 30291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e264c3f2384804bf10b53c14b3243e, State: Initialized, Role: FOLLOWER
I20260808 00:36:46.602753 30291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e264c3f2384804bf10b53c14b3243e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43621 } }
I20260808 00:36:46.602837 30291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:46.602882 30291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:46.602932 30291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:46.603489 30291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e264c3f2384804bf10b53c14b3243e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43621 } }
I20260808 00:36:46.603614 30291 leader_election.cc:302] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63e264c3f2384804bf10b53c14b3243e; no voters:
I20260808 00:36:46.603772 30291 leader_election.cc:288] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:46.604007 30306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e264c3f2384804bf10b53c14b3243e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43621 } } }
I20260808 00:36:46.604038 30291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:46.604068 30306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:36:46.604575 30305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:46.604662 30305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [term 1 LEADER]: Becoming Leader. State: Replica: 63e264c3f2384804bf10b53c14b3243e, State: Running, Role: LEADER
I20260808 00:36:46.604789 30305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e264c3f2384804bf10b53c14b3243e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43621 } }
I20260808 00:36:46.605191 30306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63e264c3f2384804bf10b53c14b3243e. Latest consensus state: current_term: 1 leader_uuid: "63e264c3f2384804bf10b53c14b3243e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e264c3f2384804bf10b53c14b3243e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43621 } } }
I20260808 00:36:46.605275 30306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:46.605501 30321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:46.606103 30321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:46.607707 30321 catalog_manager.cc:1347] Generated new cluster ID: 31c19a82a791455e996fef23e4673edc
I20260808 00:36:46.607766 30321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:46.639024 30321 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:46.639251 30321 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:46.652895 30321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63e264c3f2384804bf10b53c14b3243e: Generated new TSK 0
I20260808 00:36:46.653244 30321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:46.659161 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 30191
2026-08-08T00:36:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:46 dist-test-slave-44d2 krb5kdc[30329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:46 dist-test-slave-44d2 krb5kdc[30329](info): set up 2 sockets
Aug 08 00:36:46 dist-test-slave-44d2 krb5kdc[30329](info): commencing operation
krb5kdc: starting...
W20260808 00:36:47.982825 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:48 dist-test-slave-44d2 krb5kdc[30329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:48.120699 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:48.205627 30348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:48.205842 30348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:48.205865 30348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:48.207122 30348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:48.207171 30348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:48.207186 30348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:48.207199 30348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:48.207211 30348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:48.208741 30348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:48.209002 30348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:48.209240 30348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:48.214185 30355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:48.214180 30357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:48.214217 30354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:48.214366 30348 server_base.cc:1034] running on GCE node
I20260808 00:36:48.214592 30348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:48.214875 30348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:48.216037 30348 hybrid_clock.cc:648] HybridClock initialized: now 1786149408216012 us; error 47 us; skew 500 ppm
Aug 08 00:36:48 dist-test-slave-44d2 krb5kdc[30329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149408, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:48.218586 30348 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:48.219038 30348 webserver.cc:460] Webserver started at http://127.0.0.1:33677/ using document root <none> and password file <none>
I20260808 00:36:48.219270 30348 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:48.219317 30348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:48.219450 30348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:48.220474 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "cd191bbd96564f1886051255e55577a8"
format_stamp: "Formatted at 2026-08-08 00:36:48 on dist-test-slave-44d2"
server_key: "df8b6878a40917060b3e9d2976b427f4"
server_key_iv: "eb2346b21f6da6458c79ce06fdeaadfa"
server_key_version: "encryptionkey@0"
I20260808 00:36:48.220888 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "cd191bbd96564f1886051255e55577a8"
format_stamp: "Formatted at 2026-08-08 00:36:48 on dist-test-slave-44d2"
server_key: "df8b6878a40917060b3e9d2976b427f4"
server_key_iv: "eb2346b21f6da6458c79ce06fdeaadfa"
server_key_version: "encryptionkey@0"
I20260808 00:36:48.222205 30348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:48.222927 30369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:48.223102 30348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:36:48.223204 30348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "cd191bbd96564f1886051255e55577a8"
format_stamp: "Formatted at 2026-08-08 00:36:48 on dist-test-slave-44d2"
server_key: "df8b6878a40917060b3e9d2976b427f4"
server_key_iv: "eb2346b21f6da6458c79ce06fdeaadfa"
server_key_version: "encryptionkey@0"
I20260808 00:36:48.223276 30348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:48.235062 30348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:48.237468 30348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:48.237654 30348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:48.242008 30348 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45319
I20260808 00:36:48.242072 30448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45319 every 8 connection(s)
I20260808 00:36:48.242455 30348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:48.243474 30450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:48.246646 30450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8: Bootstrap starting.
I20260808 00:36:48.246897 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 30348
I20260808 00:36:48.247021 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:48.247236 32587 external_mini_cluster.cc:1428] Setting key f5a142528e233d2c2114b7035c9e0dde
I20260808 00:36:48.247471 30450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:48.247849 30450 log.cc:824] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:48.249066 30450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8: No bootstrap required, opened a new log
I20260808 00:36:48.250775 30450 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd191bbd96564f1886051255e55577a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260808 00:36:48.250929 30450 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:48.250954 30450 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd191bbd96564f1886051255e55577a8, State: Initialized, Role: FOLLOWER
I20260808 00:36:48.251093 30450 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd191bbd96564f1886051255e55577a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260808 00:36:48.251183 30450 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:48.251225 30450 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:48.251277 30450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:48.252030 30450 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd191bbd96564f1886051255e55577a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260808 00:36:48.252144 30450 leader_election.cc:302] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd191bbd96564f1886051255e55577a8; no voters:
I20260808 00:36:48.252311 30450 leader_election.cc:288] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:48.252367 30458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:48.252548 30458 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [term 1 LEADER]: Becoming Leader. State: Replica: cd191bbd96564f1886051255e55577a8, State: Running, Role: LEADER
I20260808 00:36:48.252631 30450 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:48.252753 30458 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd191bbd96564f1886051255e55577a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260808 00:36:48.253170 30458 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd191bbd96564f1886051255e55577a8. Latest consensus state: current_term: 1 leader_uuid: "cd191bbd96564f1886051255e55577a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd191bbd96564f1886051255e55577a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } } }
I20260808 00:36:48.253399 30458 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:48.253324 30460 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd191bbd96564f1886051255e55577a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd191bbd96564f1886051255e55577a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } } }
I20260808 00:36:48.253846 30460 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:48.254078 30475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:48.255025 30475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:36:48 dist-test-slave-44d2 krb5kdc[30329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:36:48.256924 30475 catalog_manager.cc:1347] Generated new cluster ID: 4f7c8208294b4232899218f883c93fb5
I20260808 00:36:48.256980 30475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:48.260727 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:48.247873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:43475 (local address 127.0.0.1:45319)
0808 00:36:48.248103 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:48.248114 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:48.248151 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:36:48.248741 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:48.248750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:48.248803 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:48.248923 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:48.249852 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:48.251462 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:48.252621 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:48.253281 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:48.253381 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:48.256868 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:48.256872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:48.256873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:48.256886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:48.258941 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:48.259383 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:48.259387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:48.259387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:48.259493 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:48.259713 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:48.259714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:48.259714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:48.259998 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:48.260124 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:48.260409 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 00:36:48.260528 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
I20260808 00:36:48.262111 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:48.272017 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:48.264805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:35843 (local address 127.0.0.1:45319)
0808 00:36:48.264843 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:48.264844 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:48.264952 (+ 108us) server_negotiation.cc:407] Connection header received
0808 00:36:48.265032 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:48.265033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:48.265042 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:48.265069 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:48.265723 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:48.266463 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:48.267498 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:48.267809 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:48.267844 (+ 35us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:48.269333 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:48.269334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:48.269334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:48.269341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:48.270849 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:48.271234 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:48.271235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:48.271235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:48.271282 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:48.271487 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:48.271488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:48.271488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:48.271601 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:48.271670 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:48.271867 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 00:36:48.271928 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:36:48.276202 30475 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:48.277046 30475 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:48.280468 30475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd191bbd96564f1886051255e55577a8: Generated new TSK 0
I20260808 00:36:48.280762 30475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:48.288379 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 30348
W20260808 00:36:48.303279 30326 connection.cc:535] client connection to 127.0.0.1:45319 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45319): Connection reset by peer (error 104)
2026-08-08T00:36:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:48 dist-test-slave-44d2 krb5kdc[30499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:48 dist-test-slave-44d2 krb5kdc[30499](info): set up 2 sockets
Aug 08 00:36:48 dist-test-slave-44d2 krb5kdc[30499](info): commencing operation
krb5kdc: starting...
W20260808 00:36:49.548285 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:49 dist-test-slave-44d2 krb5kdc[30499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:49.679590 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:49.764737 30514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:49.765065 30514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:49.765112 30514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:49.766400 30514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:49.766453 30514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:49.766469 30514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:49.766481 30514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:49.766499 30514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:49.768196 30514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:49.768487 30514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:49.768795 30514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:49.773830 30523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:49.773874 30519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:49.773885 30521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:49.774039 30514 server_base.cc:1034] running on GCE node
I20260808 00:36:49.774346 30514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:49.774653 30514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:49.775799 30514 hybrid_clock.cc:648] HybridClock initialized: now 1786149409775810 us; error 88 us; skew 500 ppm
Aug 08 00:36:49 dist-test-slave-44d2 krb5kdc[30499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149409, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:49.783676 30514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:49.784221 30514 webserver.cc:460] Webserver started at http://127.0.0.1:45915/ using document root <none> and password file <none>
I20260808 00:36:49.784533 30514 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:49.784616 30514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:49.784785 30514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:49.785917 30514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "d8a04cfc736342cbbedcbb26c871512b"
format_stamp: "Formatted at 2026-08-08 00:36:49 on dist-test-slave-44d2"
server_key: "5d444a6a5bcd8f6e59fcf49e92c0e48a"
server_key_iv: "b852cf3a606aa0d68cf61416f9bcea83"
server_key_version: "encryptionkey@0"
I20260808 00:36:49.786306 30514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "d8a04cfc736342cbbedcbb26c871512b"
format_stamp: "Formatted at 2026-08-08 00:36:49 on dist-test-slave-44d2"
server_key: "5d444a6a5bcd8f6e59fcf49e92c0e48a"
server_key_iv: "b852cf3a606aa0d68cf61416f9bcea83"
server_key_version: "encryptionkey@0"
I20260808 00:36:49.787720 30514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:49.788623 30530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:49.788831 30514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:49.788942 30514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "d8a04cfc736342cbbedcbb26c871512b"
format_stamp: "Formatted at 2026-08-08 00:36:49 on dist-test-slave-44d2"
server_key: "5d444a6a5bcd8f6e59fcf49e92c0e48a"
server_key_iv: "b852cf3a606aa0d68cf61416f9bcea83"
server_key_version: "encryptionkey@0"
I20260808 00:36:49.789044 30514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:49.800072 30514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:49.802196 30514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:49.802388 30514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:49.806885 30514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44107
I20260808 00:36:49.806921 30604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44107 every 8 connection(s)
I20260808 00:36:49.807344 30514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:49.808284 30606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:49.810664 30606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b: Bootstrap starting.
I20260808 00:36:49.811231 30606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:49.811479 30606 log.cc:824] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:49.812191 30606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b: No bootstrap required, opened a new log
I20260808 00:36:49.813774 30606 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a04cfc736342cbbedcbb26c871512b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44107 } }
I20260808 00:36:49.813915 30606 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:49.813939 30606 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a04cfc736342cbbedcbb26c871512b, State: Initialized, Role: FOLLOWER
I20260808 00:36:49.814054 30606 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a04cfc736342cbbedcbb26c871512b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44107 } }
I20260808 00:36:49.814109 30606 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:49.814131 30606 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:49.814157 30606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:49.814656 30606 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a04cfc736342cbbedcbb26c871512b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44107 } }
I20260808 00:36:49.814744 30606 leader_election.cc:302] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8a04cfc736342cbbedcbb26c871512b; no voters:
I20260808 00:36:49.814908 30606 leader_election.cc:288] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:49.814942 30609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:49.815069 30609 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [term 1 LEADER]: Becoming Leader. State: Replica: d8a04cfc736342cbbedcbb26c871512b, State: Running, Role: LEADER
I20260808 00:36:49.815204 30606 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:49.815233 30609 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a04cfc736342cbbedcbb26c871512b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44107 } }
I20260808 00:36:49.815670 30610 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8a04cfc736342cbbedcbb26c871512b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a04cfc736342cbbedcbb26c871512b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44107 } } }
I20260808 00:36:49.815901 30610 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:49.815933 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 30514
I20260808 00:36:49.816095 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:49.815832 30613 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8a04cfc736342cbbedcbb26c871512b. Latest consensus state: current_term: 1 leader_uuid: "d8a04cfc736342cbbedcbb26c871512b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8a04cfc736342cbbedcbb26c871512b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44107 } } }
I20260808 00:36:49.816126 30613 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:49.816246 30615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:49.816339 32587 external_mini_cluster.cc:1428] Setting key 776e604071e7a54473d6deb4b8eacea0
I20260808 00:36:49.816807 30615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:49.818560 30615 catalog_manager.cc:1347] Generated new cluster ID: 361df7ff891d4742a4fdfac888641cfc
I20260808 00:36:49.818617 30615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:49.823647 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:49.817089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:56119 (local address 127.0.0.1:44107)
0808 00:36:49.817571 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:49.817580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:49.817614 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:36:49.817736 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:49.817743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:49.817802 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:49.817947 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:49.818289 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:49.819719 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:49.820867 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:49.821490 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:49.821596 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:49.822401 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:49.822405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:49.822405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:49.822625 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:49.822663 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:49.822676 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:49.822757 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:49.822879 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 00:36:49.823261 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":79,"threads_started":1}
I20260808 00:36:49.824548 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:49.829773 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:49.826022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:57713 (local address 127.0.0.1:44107)
0808 00:36:49.826053 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:49.826055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:49.826295 (+ 240us) server_negotiation.cc:407] Connection header received
0808 00:36:49.826375 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:49.826376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:49.826385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:49.826411 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:49.826625 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:49.827348 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:49.828362 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:49.828705 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:49.828742 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:49.829506 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:49.829508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:49.829508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:49.829513 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:49.829527 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:49.829531 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:49.829565 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:49.829687 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 00:36:49.829699 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 00:36:49.831902 30615 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:49.832398 30615 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:49.836359 30615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8a04cfc736342cbbedcbb26c871512b: Generated new TSK 0
I20260808 00:36:49.836594 30615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:49.846464 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 30514
W20260808 00:36:49.859368 30496 connection.cc:535] client connection to 127.0.0.1:44107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44107): Connection reset by peer (error 104)
2026-08-08T00:36:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:49 dist-test-slave-44d2 krb5kdc[30650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:49 dist-test-slave-44d2 krb5kdc[30650](info): set up 2 sockets
Aug 08 00:36:49 dist-test-slave-44d2 krb5kdc[30650](info): commencing operation
krb5kdc: starting...
W20260808 00:36:51.082757 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:51 dist-test-slave-44d2 krb5kdc[30650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:51.220640 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38043
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:51.311674 30666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:51.311920 30666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:51.311949 30666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:51.313194 30666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:51.313252 30666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:51.313269 30666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:51.313284 30666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:51.313299 30666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:51.314793 30666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:51.315027 30666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:51.315224 30666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:51.320961 30673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:51.321067 30676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:51.321098 30666 server_base.cc:1034] running on GCE node
W20260808 00:36:51.320986 30672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:51.321446 30666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:51.321736 30666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:51.322973 30666 hybrid_clock.cc:648] HybridClock initialized: now 1786149411322950 us; error 40 us; skew 500 ppm
Aug 08 00:36:51 dist-test-slave-44d2 krb5kdc[30650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149411, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:51.325791 30666 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:51.326373 30666 webserver.cc:460] Webserver started at http://127.0.0.1:32961/ using document root <none> and password file <none>
I20260808 00:36:51.326615 30666 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:51.326663 30666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:51.326841 30666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:51.327970 30666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "91b8fe3fb3d94144a130df3624d70a9d"
format_stamp: "Formatted at 2026-08-08 00:36:51 on dist-test-slave-44d2"
server_key: "1b78902612a33ec23acf70572523c902"
server_key_iv: "207e3dd8a688c772d452a7fd115b4c40"
server_key_version: "encryptionkey@0"
I20260808 00:36:51.328357 30666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "91b8fe3fb3d94144a130df3624d70a9d"
format_stamp: "Formatted at 2026-08-08 00:36:51 on dist-test-slave-44d2"
server_key: "1b78902612a33ec23acf70572523c902"
server_key_iv: "207e3dd8a688c772d452a7fd115b4c40"
server_key_version: "encryptionkey@0"
I20260808 00:36:51.329849 30666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:51.330729 30688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:51.330978 30666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:51.331099 30666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "91b8fe3fb3d94144a130df3624d70a9d"
format_stamp: "Formatted at 2026-08-08 00:36:51 on dist-test-slave-44d2"
server_key: "1b78902612a33ec23acf70572523c902"
server_key_iv: "207e3dd8a688c772d452a7fd115b4c40"
server_key_version: "encryptionkey@0"
I20260808 00:36:51.331180 30666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:51.334918 30666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:51.337361 30666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:51.337531 30666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:51.341784 30666 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38043
I20260808 00:36:51.341814 30754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38043 every 8 connection(s)
I20260808 00:36:51.342211 30666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:51.343276 30755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:51.346140 30755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d: Bootstrap starting.
I20260808 00:36:51.346745 30755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:51.347038 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 30666
I20260808 00:36:51.347070 30755 log.cc:824] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:51.347177 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:51.347420 32587 external_mini_cluster.cc:1428] Setting key 3152ba0c388914e810e55a7d0f09e328
I20260808 00:36:51.348359 30755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d: No bootstrap required, opened a new log
I20260808 00:36:51.350204 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:51.347988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:34907 (local address 127.0.0.1:38043)
0808 00:36:51.348194 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:51.348203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:51.348234 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:36:51.348842 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:51.348852 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:51.348892 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:51.349051 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:51.349739 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:51.349743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:51.349743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:51.349747 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:51.349797 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:51.349812 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:51.349914 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:51.349944 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:36:51.349992 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260808 00:36:51.351064 30755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b8fe3fb3d94144a130df3624d70a9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38043 } }
I20260808 00:36:51.351239 30755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:51.351284 30755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b8fe3fb3d94144a130df3624d70a9d, State: Initialized, Role: FOLLOWER
I20260808 00:36:51.351361 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:51.351454 30755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b8fe3fb3d94144a130df3624d70a9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38043 } }
I20260808 00:36:51.351539 30755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:51.351568 30755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:51.351605 30755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:51.352337 30755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b8fe3fb3d94144a130df3624d70a9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38043 } }
I20260808 00:36:51.352442 30755 leader_election.cc:302] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91b8fe3fb3d94144a130df3624d70a9d; no voters:
I20260808 00:36:51.352667 30755 leader_election.cc:288] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:51.352707 30772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:51.352900 30772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [term 1 LEADER]: Becoming Leader. State: Replica: 91b8fe3fb3d94144a130df3624d70a9d, State: Running, Role: LEADER
I20260808 00:36:51.352934 30755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:51.353004 30772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b8fe3fb3d94144a130df3624d70a9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38043 } }
I20260808 00:36:51.353457 30772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91b8fe3fb3d94144a130df3624d70a9d. Latest consensus state: current_term: 1 leader_uuid: "91b8fe3fb3d94144a130df3624d70a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b8fe3fb3d94144a130df3624d70a9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38043 } } }
I20260808 00:36:51.353704 30772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:51.353621 30779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91b8fe3fb3d94144a130df3624d70a9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b8fe3fb3d94144a130df3624d70a9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38043 } } }
I20260808 00:36:51.354067 30779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:51.354223 30789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:51.354746 30789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:51.355504 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:51.353138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:50827 (local address 127.0.0.1:38043)
0808 00:36:51.353175 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:51.353177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:51.354551 (+ 1374us) server_negotiation.cc:407] Connection header received
0808 00:36:51.354609 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:51.354610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:51.354615 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:51.354664 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:51.355269 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:51.355270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:51.355270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:51.355275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:51.355289 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:51.355293 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:51.355346 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:51.355412 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 00:36:51.355427 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:36:51.356775 30789 catalog_manager.cc:1347] Generated new cluster ID: ae634eefa0ab4bc89ca87994885af486
I20260808 00:36:51.356829 30789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:51.364452 30789 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:51.364651 30789 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:51.367650 30789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91b8fe3fb3d94144a130df3624d70a9d: Generated new TSK 0
I20260808 00:36:51.367846 30789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:51.371892 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 30666
2026-08-08T00:36:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:51 dist-test-slave-44d2 krb5kdc[30809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:51 dist-test-slave-44d2 krb5kdc[30809](info): set up 2 sockets
Aug 08 00:36:51 dist-test-slave-44d2 krb5kdc[30809](info): commencing operation
krb5kdc: starting...
W20260808 00:36:52.688320 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:52 dist-test-slave-44d2 krb5kdc[30809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:52.817922 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:52.907534 30826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:52.907790 30826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:52.907829 30826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:52.909065 30826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:52.909142 30826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:52.909166 30826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:52.909185 30826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:52.909215 30826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:52.910741 30826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:52.911072 30826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:52.911324 30826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:36:52.916550 30826 server_base.cc:1034] running on GCE node
W20260808 00:36:52.916522 30833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:52.916519 30837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:52.916486 30834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:52.917043 30826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:52.917340 30826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:52.918514 30826 hybrid_clock.cc:648] HybridClock initialized: now 1786149412918497 us; error 37 us; skew 500 ppm
Aug 08 00:36:52 dist-test-slave-44d2 krb5kdc[30809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149412, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:52.921144 30826 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:52.921542 30826 webserver.cc:460] Webserver started at http://127.0.0.1:34493/ using document root <none> and password file <none>
I20260808 00:36:52.921749 30826 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:52.921793 30826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:52.921882 30826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:52.922896 30826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "571620f3b150417795b179d45dd7e375"
format_stamp: "Formatted at 2026-08-08 00:36:52 on dist-test-slave-44d2"
server_key: "afb44ed64f3a8b83128cc1eedd07d2f9"
server_key_iv: "2beb9fd307be6e02b5aba965c661cce4"
server_key_version: "encryptionkey@0"
I20260808 00:36:52.923198 30826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "571620f3b150417795b179d45dd7e375"
format_stamp: "Formatted at 2026-08-08 00:36:52 on dist-test-slave-44d2"
server_key: "afb44ed64f3a8b83128cc1eedd07d2f9"
server_key_iv: "2beb9fd307be6e02b5aba965c661cce4"
server_key_version: "encryptionkey@0"
I20260808 00:36:52.924612 30826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:36:52.925396 30846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:52.925619 30826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:52.925717 30826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "571620f3b150417795b179d45dd7e375"
format_stamp: "Formatted at 2026-08-08 00:36:52 on dist-test-slave-44d2"
server_key: "afb44ed64f3a8b83128cc1eedd07d2f9"
server_key_iv: "2beb9fd307be6e02b5aba965c661cce4"
server_key_version: "encryptionkey@0"
I20260808 00:36:52.925769 30826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:52.952039 30826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:52.954572 30826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:52.954696 30826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:52.958815 30826 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45861
I20260808 00:36:52.958904 30924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45861 every 8 connection(s)
I20260808 00:36:52.959221 30826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:52.960175 30925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:52.962560 30925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375: Bootstrap starting.
I20260808 00:36:52.963176 30925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:52.963483 30925 log.cc:824] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:52.964313 30925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375: No bootstrap required, opened a new log
I20260808 00:36:52.964399 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 30826
I20260808 00:36:52.964584 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:52.964803 32587 external_mini_cluster.cc:1428] Setting key 859e64fc6510a1a938a6ebc4f72df8d3
I20260808 00:36:52.966784 30925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571620f3b150417795b179d45dd7e375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45861 } }
I20260808 00:36:52.967010 30925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:52.967048 30925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571620f3b150417795b179d45dd7e375, State: Initialized, Role: FOLLOWER
I20260808 00:36:52.967172 30925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571620f3b150417795b179d45dd7e375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45861 } }
I20260808 00:36:52.967264 30925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:52.967299 30925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:52.967391 30925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:52.967971 30925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571620f3b150417795b179d45dd7e375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45861 } }
I20260808 00:36:52.968112 30925 leader_election.cc:302] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 571620f3b150417795b179d45dd7e375; no voters:
I20260808 00:36:52.968272 30925 leader_election.cc:288] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:52.968359 30931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:52.968564 30925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:52.968568 30931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [term 1 LEADER]: Becoming Leader. State: Replica: 571620f3b150417795b179d45dd7e375, State: Running, Role: LEADER
I20260808 00:36:52.968783 30931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571620f3b150417795b179d45dd7e375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45861 } }
I20260808 00:36:52.969302 30931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 571620f3b150417795b179d45dd7e375. Latest consensus state: current_term: 1 leader_uuid: "571620f3b150417795b179d45dd7e375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571620f3b150417795b179d45dd7e375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45861 } } }
I20260808 00:36:52.969312 30933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "571620f3b150417795b179d45dd7e375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571620f3b150417795b179d45dd7e375" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45861 } } }
I20260808 00:36:52.969455 30933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:52.969671 30931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:52.969678 30942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:52.970518 30942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:52.972411 30942 catalog_manager.cc:1347] Generated new cluster ID: f07e1fb204254c22b7cb12a656f899fe
I20260808 00:36:52.972471 30942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:36:52 dist-test-slave-44d2 krb5kdc[30809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:36:52.979492 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:52.965372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:37161 (local address 127.0.0.1:45861)
0808 00:36:52.965582 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:52.965592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:52.965627 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:36:52.967002 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:52.967011 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:52.967084 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:52.967245 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:52.968072 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:52.969604 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:52.970747 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:52.971360 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:52.971498 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:52.974611 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:52.974616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:52.974616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:52.974630 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:52.977574 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:52.978104 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:52.978108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:52.978108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:52.978226 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:52.978491 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:52.978492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:52.978493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:52.978810 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:52.978992 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:52.979149 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 00:36:52.979257 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260808 00:36:52.980546 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:52.989897 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:52.982272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:35975 (local address 127.0.0.1:45861)
0808 00:36:52.982308 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:52.982310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:52.982318 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:36:52.982339 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:52.982340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:52.982358 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:52.982426 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:52.983083 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:52.983844 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:52.984960 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:52.985288 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:52.985361 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:52.986810 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:52.986811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:52.986812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:52.986819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:52.988534 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:52.989010 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:52.989011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:52.989011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:52.989077 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:52.989321 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:52.989322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:52.989322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:52.989459 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:52.989562 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:52.989728 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:36:52.989797 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:36:52.994396 30942 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:52.994952 30942 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:53.007412 30942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 571620f3b150417795b179d45dd7e375: Generated new TSK 0
I20260808 00:36:53.007685 30942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:53.013140 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 30826
W20260808 00:36:53.029573 30805 connection.cc:535] client connection to 127.0.0.1:45861 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45861): Connection reset by peer (error 104)
2026-08-08T00:36:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1633 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:53 dist-test-slave-44d2 krb5kdc[30966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:53 dist-test-slave-44d2 krb5kdc[30966](info): set up 2 sockets
Aug 08 00:36:53 dist-test-slave-44d2 krb5kdc[30966](info): commencing operation
krb5kdc: starting...
W20260808 00:36:54.266146 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:54 dist-test-slave-44d2 krb5kdc[30966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:54.400450 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:54.490916 30981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:54.491184 30981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:54.491230 30981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:54.492413 30981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:54.492492 30981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:54.492569 30981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:54.492596 30981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:54.492672 30981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:54.494307 30981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45493
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:54.494658 30981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:54.494932 30981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:54.500519 30989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:54.500514 30987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:54.500550 30991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:54.500758 30981 server_base.cc:1034] running on GCE node
I20260808 00:36:54.501150 30981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:54.501412 30981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:54.502583 30981 hybrid_clock.cc:648] HybridClock initialized: now 1786149414502568 us; error 47 us; skew 500 ppm
Aug 08 00:36:54 dist-test-slave-44d2 krb5kdc[30966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149414, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:54.505316 30981 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:54.505764 30981 webserver.cc:460] Webserver started at http://127.0.0.1:45485/ using document root <none> and password file <none>
I20260808 00:36:54.506026 30981 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:54.506098 30981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:54.506260 30981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:54.507357 30981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "9d208fa5374f4bdab06ea2fd78a1ba19"
format_stamp: "Formatted at 2026-08-08 00:36:54 on dist-test-slave-44d2"
server_key: "ce58574e00a27d21bf11f9c7346ca0e8"
server_key_iv: "7a0c32d71537e2c8623e063bfc4edc5e"
server_key_version: "encryptionkey@0"
I20260808 00:36:54.507720 30981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "9d208fa5374f4bdab06ea2fd78a1ba19"
format_stamp: "Formatted at 2026-08-08 00:36:54 on dist-test-slave-44d2"
server_key: "ce58574e00a27d21bf11f9c7346ca0e8"
server_key_iv: "7a0c32d71537e2c8623e063bfc4edc5e"
server_key_version: "encryptionkey@0"
I20260808 00:36:54.509173 30981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:36:54.509984 31001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:54.510224 30981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:54.510334 30981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "9d208fa5374f4bdab06ea2fd78a1ba19"
format_stamp: "Formatted at 2026-08-08 00:36:54 on dist-test-slave-44d2"
server_key: "ce58574e00a27d21bf11f9c7346ca0e8"
server_key_iv: "7a0c32d71537e2c8623e063bfc4edc5e"
server_key_version: "encryptionkey@0"
I20260808 00:36:54.510409 30981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:54.530910 30981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:54.533032 30981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:54.533212 30981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:54.537623 30981 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45493
I20260808 00:36:54.537663 31081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45493 every 8 connection(s)
I20260808 00:36:54.538069 30981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:54.539021 31083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:54.541749 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19: Bootstrap starting.
I20260808 00:36:54.542479 31083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:54.542807 31083 log.cc:824] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:54.543613 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19: No bootstrap required, opened a new log
I20260808 00:36:54.545375 31083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45493 } }
I20260808 00:36:54.545524 31083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:54.545548 31083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d208fa5374f4bdab06ea2fd78a1ba19, State: Initialized, Role: FOLLOWER
I20260808 00:36:54.545727 31083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45493 } }
I20260808 00:36:54.545790 31083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:54.545812 31083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:54.545886 31083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:54.546465 31083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45493 } }
I20260808 00:36:54.546583 31083 leader_election.cc:302] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d208fa5374f4bdab06ea2fd78a1ba19; no voters:
I20260808 00:36:54.546804 31083 leader_election.cc:288] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:54.546831 31088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:54.546881 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 30981
I20260808 00:36:54.546962 31088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [term 1 LEADER]: Becoming Leader. State: Replica: 9d208fa5374f4bdab06ea2fd78a1ba19, State: Running, Role: LEADER
I20260808 00:36:54.547004 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:54.547108 31083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:54.547132 31088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45493 } }
I20260808 00:36:54.547209 32587 external_mini_cluster.cc:1428] Setting key e4727d642a88570b953bd3ed1e468ac2
I20260808 00:36:54.547749 31089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45493 } } }
I20260808 00:36:54.547842 31089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:54.548400 31103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:54.548460 31092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d208fa5374f4bdab06ea2fd78a1ba19. Latest consensus state: current_term: 1 leader_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d208fa5374f4bdab06ea2fd78a1ba19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45493 } } }
I20260808 00:36:54.548811 31092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:54.549126 31103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:54.550772 31103 catalog_manager.cc:1347] Generated new cluster ID: 382f878a97ad4f5a8d63b5579587fd7a
I20260808 00:36:54.550825 31103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:54.553730 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:54.547899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:51059 (local address 127.0.0.1:45493)
0808 00:36:54.548085 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:54.548096 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:54.548247 (+ 151us) server_negotiation.cc:407] Connection header received
0808 00:36:54.548389 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:54.548398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:54.548471 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:54.548746 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:54.549117 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:54.550338 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:54.551469 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:54.552075 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:54.552187 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:54.553023 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:54.553027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:54.553028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:54.553237 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:54.553277 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:54.553292 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:54.553375 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:54.553487 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:36:54.553521 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260808 00:36:54.554760 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:54.560005 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:54.556473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:54589 (local address 127.0.0.1:45493)
0808 00:36:54.556543 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:54.556553 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:54.556562 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:36:54.556619 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:54.556619 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:54.556630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:54.556702 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:54.556843 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:54.557584 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:54.558596 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:54.558917 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:54.558995 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:54.559719 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:54.559721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:54.559721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:54.559726 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:54.559742 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:54.559747 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:54.559814 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:54.559897 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:36:54.559911 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:36:54.561623 31103 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:54.562304 31103 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:54.572033 31103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d208fa5374f4bdab06ea2fd78a1ba19: Generated new TSK 0
I20260808 00:36:54.572232 31103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:54.576107 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 30981
W20260808 00:36:54.590798 30964 connection.cc:535] client connection to 127.0.0.1:45493 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45493): Connection reset by peer (error 104)
2026-08-08T00:36:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:54 dist-test-slave-44d2 krb5kdc[31131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:54 dist-test-slave-44d2 krb5kdc[31131](info): set up 2 sockets
Aug 08 00:36:54 dist-test-slave-44d2 krb5kdc[31131](info): commencing operation
krb5kdc: starting...
W20260808 00:36:55.802561 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:55 dist-test-slave-44d2 krb5kdc[31131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:36:55.933561 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:56.020785 31148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:56.021030 31148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:56.021062 31148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:56.022396 31148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:56.022446 31148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:56.022462 31148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:56.022476 31148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:56.022536 31148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:56.024222 31148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:56.024528 31148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:56.024811 31148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:56.030555 31160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:56.030584 31156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:56.030553 31157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:56.030776 31148 server_base.cc:1034] running on GCE node
I20260808 00:36:56.031051 31148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:56.031338 31148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:56.032546 31148 hybrid_clock.cc:648] HybridClock initialized: now 1786149416032522 us; error 31 us; skew 500 ppm
Aug 08 00:36:56 dist-test-slave-44d2 krb5kdc[31131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149416, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:56.039124 31148 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:36:56.039582 31148 webserver.cc:460] Webserver started at http://127.0.0.1:37073/ using document root <none> and password file <none>
I20260808 00:36:56.039813 31148 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:56.039860 31148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:56.039978 31148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:56.041179 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "52d783491d36427abd251aed124ce43e"
format_stamp: "Formatted at 2026-08-08 00:36:56 on dist-test-slave-44d2"
server_key: "fa6b52133521941cc006199eb6008bb7"
server_key_iv: "22d00e223923093bb064e9038a4c2465"
server_key_version: "encryptionkey@0"
I20260808 00:36:56.041591 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "52d783491d36427abd251aed124ce43e"
format_stamp: "Formatted at 2026-08-08 00:36:56 on dist-test-slave-44d2"
server_key: "fa6b52133521941cc006199eb6008bb7"
server_key_iv: "22d00e223923093bb064e9038a4c2465"
server_key_version: "encryptionkey@0"
I20260808 00:36:56.043007 31148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:36:56.043774 31169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:56.043970 31148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:56.044075 31148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "52d783491d36427abd251aed124ce43e"
format_stamp: "Formatted at 2026-08-08 00:36:56 on dist-test-slave-44d2"
server_key: "fa6b52133521941cc006199eb6008bb7"
server_key_iv: "22d00e223923093bb064e9038a4c2465"
server_key_version: "encryptionkey@0"
I20260808 00:36:56.044165 31148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:56.047967 31148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:56.050246 31148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:56.050447 31148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:56.055197 31148 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39225
I20260808 00:36:56.055253 31241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39225 every 8 connection(s)
I20260808 00:36:56.055652 31148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:56.056831 31243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:56.059144 31243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e: Bootstrap starting.
I20260808 00:36:56.059692 31243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:56.059937 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 31148
I20260808 00:36:56.060068 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:56.060079 31243 log.cc:824] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:56.060303 32587 external_mini_cluster.cc:1428] Setting key d04178391f0bbe36ea2c33b49c2aa19d
I20260808 00:36:56.060987 31243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e: No bootstrap required, opened a new log
I20260808 00:36:56.063205 31243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d783491d36427abd251aed124ce43e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260808 00:36:56.063374 31243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:56.063412 31243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d783491d36427abd251aed124ce43e, State: Initialized, Role: FOLLOWER
I20260808 00:36:56.063588 31243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d783491d36427abd251aed124ce43e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260808 00:36:56.063706 31243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:56.063746 31243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:56.063808 31243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:56.063901 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:56.060949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:37989 (local address 127.0.0.1:39225)
0808 00:36:56.061340 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:56.061349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:56.061379 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:36:56.062363 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:56.062373 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:56.062409 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:56.062563 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:56.063386 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:56.063390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:56.063391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:56.063396 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:56.063446 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:56.063461 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:56.063527 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:56.063626 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 00:36:56.063671 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":78,"threads_started":1}
I20260808 00:36:56.064927 31243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d783491d36427abd251aed124ce43e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260808 00:36:56.065055 31243 leader_election.cc:302] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52d783491d36427abd251aed124ce43e; no voters:
I20260808 00:36:56.065150 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:56.065233 31243 leader_election.cc:288] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:56.065327 31252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:56.065505 31243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:56.065546 31252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [term 1 LEADER]: Becoming Leader. State: Replica: 52d783491d36427abd251aed124ce43e, State: Running, Role: LEADER
I20260808 00:36:56.065652 31252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d783491d36427abd251aed124ce43e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } }
I20260808 00:36:56.065985 31255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52d783491d36427abd251aed124ce43e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d783491d36427abd251aed124ce43e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } } }
I20260808 00:36:56.066071 31255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:56.066318 31256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52d783491d36427abd251aed124ce43e. Latest consensus state: current_term: 1 leader_uuid: "52d783491d36427abd251aed124ce43e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d783491d36427abd251aed124ce43e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39225 } } }
I20260808 00:36:56.066479 31256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:56.066653 31262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:56.067385 31262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:56.068157 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:56.066824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.58:59311 (local address 127.0.0.1:39225)
0808 00:36:56.066863 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:56.066865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:56.067149 (+ 284us) server_negotiation.cc:407] Connection header received
0808 00:36:56.067224 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:56.067225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:56.067230 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:56.067276 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:56.067937 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:56.067938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:56.067938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:56.067944 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:56.067959 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:56.067964 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:56.068018 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:56.068078 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 00:36:56.068093 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:36:56.069561 31262 catalog_manager.cc:1347] Generated new cluster ID: 9eaa3a334b8f4ffb859ca9a38555177b
I20260808 00:36:56.069618 31262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:56.099932 31262 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:56.100204 31262 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:56.110059 31262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52d783491d36427abd251aed124ce43e: Generated new TSK 0
I20260808 00:36:56.110292 31262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:56.120715 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 31148
2026-08-08T00:36:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:56 dist-test-slave-44d2 krb5kdc[31289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:56 dist-test-slave-44d2 krb5kdc[31289](info): set up 2 sockets
Aug 08 00:36:56 dist-test-slave-44d2 krb5kdc[31289](info): commencing operation
krb5kdc: starting...
W20260808 00:36:57.355463 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:57 dist-test-slave-44d2 krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:36:57.492257 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39795
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:39795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:57.587311 31305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:57.587605 31305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:57.587651 31305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:57.589053 31305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:57.589131 31305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:57.589159 31305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:57.589191 31305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:57.589226 31305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:57.590883 31305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:39795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:57.591195 31305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:57.591465 31305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:57.597072 31315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:57.597182 31317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:57.597318 31305 server_base.cc:1034] running on GCE node
W20260808 00:36:57.597072 31312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:57.597615 31305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:57.597921 31305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:57.599098 31305 hybrid_clock.cc:648] HybridClock initialized: now 1786149417599065 us; error 48 us; skew 500 ppm
Aug 08 00:36:57 dist-test-slave-44d2 krb5kdc[31289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149417, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:57.601926 31305 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:36:57.602473 31305 webserver.cc:460] Webserver started at http://127.31.210.254:39113/ using document root <none> and password file <none>
I20260808 00:36:57.602744 31305 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:57.602816 31305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:57.602973 31305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:57.604118 31305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "c99a9cf30e004fd5a38c39f3011f6539"
format_stamp: "Formatted at 2026-08-08 00:36:57 on dist-test-slave-44d2"
server_key: "b60b4a263c16be155e97033757c30d90"
server_key_iv: "98a2a3d0ca37c0bac3226714369a0899"
server_key_version: "encryptionkey@0"
I20260808 00:36:57.604563 31305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "c99a9cf30e004fd5a38c39f3011f6539"
format_stamp: "Formatted at 2026-08-08 00:36:57 on dist-test-slave-44d2"
server_key: "b60b4a263c16be155e97033757c30d90"
server_key_iv: "98a2a3d0ca37c0bac3226714369a0899"
server_key_version: "encryptionkey@0"
I20260808 00:36:57.606175 31305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:36:57.607205 31326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:57.607467 31305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:36:57.607579 31305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "c99a9cf30e004fd5a38c39f3011f6539"
format_stamp: "Formatted at 2026-08-08 00:36:57 on dist-test-slave-44d2"
server_key: "b60b4a263c16be155e97033757c30d90"
server_key_iv: "98a2a3d0ca37c0bac3226714369a0899"
server_key_version: "encryptionkey@0"
I20260808 00:36:57.607676 31305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:57.620453 31305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:57.623162 31305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:57.623356 31305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:57.627802 31305 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:39795
I20260808 00:36:57.627842 31395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:39795 every 8 connection(s)
I20260808 00:36:57.628268 31305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:57.629285 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 31305
I20260808 00:36:57.629297 31396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:57.629449 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:57.629673 32587 external_mini_cluster.cc:1428] Setting key 9c21600c163c943f74bd291d7de927ba
I20260808 00:36:57.633103 31396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539: Bootstrap starting.
I20260808 00:36:57.633771 31396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:57.634092 31396 log.cc:824] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:57.634872 31396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539: No bootstrap required, opened a new log
Aug 08 00:36:57 dist-test-slave-44d2 krb5kdc[31289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:36:57.637199 31396 raft_consensus.cc:348] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a9cf30e004fd5a38c39f3011f6539" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39795 } }
I20260808 00:36:57.637346 31396 raft_consensus.cc:374] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:57.637379 31396 raft_consensus.cc:729] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c99a9cf30e004fd5a38c39f3011f6539, State: Initialized, Role: FOLLOWER
I20260808 00:36:57.637550 31396 consensus_queue.cc:260] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a9cf30e004fd5a38c39f3011f6539" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39795 } }
I20260808 00:36:57.637624 31396 raft_consensus.cc:388] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:57.637650 31396 raft_consensus.cc:482] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:57.637723 31396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:57.638338 31396 raft_consensus.cc:504] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a9cf30e004fd5a38c39f3011f6539" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39795 } }
I20260808 00:36:57.638446 31396 leader_election.cc:302] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c99a9cf30e004fd5a38c39f3011f6539; no voters:
I20260808 00:36:57.638682 31396 leader_election.cc:288] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:57.638724 31402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:57.638975 31396 sys_catalog.cc:564] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:57.639842 31402 raft_consensus.cc:686] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [term 1 LEADER]: Becoming Leader. State: Replica: c99a9cf30e004fd5a38c39f3011f6539, State: Running, Role: LEADER
I20260808 00:36:57.640002 31402 consensus_queue.cc:237] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a9cf30e004fd5a38c39f3011f6539" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39795 } }
I20260808 00:36:57.640425 31409 sys_catalog.cc:455] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c99a9cf30e004fd5a38c39f3011f6539. Latest consensus state: current_term: 1 leader_uuid: "c99a9cf30e004fd5a38c39f3011f6539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a9cf30e004fd5a38c39f3011f6539" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39795 } } }
I20260808 00:36:57.640542 31409 sys_catalog.cc:458] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:57.640827 31403 sys_catalog.cc:455] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c99a9cf30e004fd5a38c39f3011f6539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a9cf30e004fd5a38c39f3011f6539" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39795 } } }
I20260808 00:36:57.641048 31403 sys_catalog.cc:458] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539 [sys.catalog]: This master's current role is: LEADER
W20260808 00:36:57.641662 31417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:36:57.641734 31417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:36:57.641811 31419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:57.642005 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:57.630296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49961 (local address 127.31.210.254:39795)
0808 00:36:57.630480 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:57.630496 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:57.630522 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:36:57.631008 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:57.631016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:57.631068 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:57.631186 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:57.632208 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:57.633214 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:57.634057 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:57.634590 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:57.634725 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:57.637655 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:57.637661 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:57.637662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:57.637676 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:57.639632 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:57.640216 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:57.640219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:57.640220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:57.640333 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:57.640650 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:57.640652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:57.640652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:57.640948 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:57.641548 (+ 600us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:57.641645 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 00:36:57.641753 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260808 00:36:57.642038 31419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:57.643030 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:57.643560 31419 catalog_manager.cc:1347] Generated new cluster ID: 2f2f5872ab514575b656fe596cfcf1b2
I20260808 00:36:57.643625 31419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:57.651667 31419 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:57.652115 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:57.644582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33729 (local address 127.31.210.254:39795)
0808 00:36:57.644621 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:57.644623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:57.644810 (+ 187us) server_negotiation.cc:407] Connection header received
0808 00:36:57.644841 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:57.644842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:57.644852 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:57.644921 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:57.645584 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:57.646392 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:57.647358 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:57.647678 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:57.647753 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:57.649111 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:57.649112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:57.649112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:36:57.649119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:57.650705 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:57.651129 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:57.651130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:57.651131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:57.651190 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:36:57.651460 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:36:57.651461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:36:57.651462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:36:57.651606 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:57.651716 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:57.651963 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 00:36:57.652030 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:36:57.652415 31419 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:57.654981 31419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c99a9cf30e004fd5a38c39f3011f6539: Generated new TSK 0
I20260808 00:36:57.655175 31419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:57.658236 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 31305
W20260808 00:36:57.676299 31287 connection.cc:535] client connection to 127.31.210.254:39795 recv error: Network error: failed to read from TLS socket (remote: 127.31.210.254:39795): Connection reset by peer (error 104)
2026-08-08T00:36:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:57 dist-test-slave-44d2 krb5kdc[31433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:57 dist-test-slave-44d2 krb5kdc[31433](info): set up 2 sockets
Aug 08 00:36:57 dist-test-slave-44d2 krb5kdc[31433](info): commencing operation
krb5kdc: starting...
W20260808 00:36:58.976766 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:36:59 dist-test-slave-44d2 krb5kdc[31433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:36:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:36:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:36:59.112857 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43201
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:43201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:36:59.203543 31455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:36:59.203873 31455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:36:59.203923 31455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:36:59.206061 31455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:36:59.206142 31455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:36:59.206171 31455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:36:59.206195 31455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:36:59.206264 31455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:36:59.208948 31455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:43201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:36:59.209280 31455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:36:59.209578 31455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:36:59.215764 31467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:36:59.215750 31469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:59.215917 31455 server_base.cc:1034] running on GCE node
W20260808 00:36:59.215750 31466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:36:59.216216 31455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:36:59.216534 31455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:36:59.217697 31455 hybrid_clock.cc:648] HybridClock initialized: now 1786149419217670 us; error 45 us; skew 500 ppm
Aug 08 00:36:59 dist-test-slave-44d2 krb5kdc[31433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149419, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:36:59.220402 31455 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:36:59.220868 31455 webserver.cc:460] Webserver started at http://127.31.210.254:36277/ using document root <none> and password file <none>
I20260808 00:36:59.221133 31455 fs_manager.cc:359] Metadata directory not provided
I20260808 00:36:59.221180 31455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:36:59.221277 31455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:36:59.222334 31455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "05854d26c9e843edab10157224ceed76"
format_stamp: "Formatted at 2026-08-08 00:36:59 on dist-test-slave-44d2"
server_key: "94f9d73916cc45f8029f9e9502bc731f"
server_key_iv: "a2a340d622eac1aea565aa2293c2626b"
server_key_version: "encryptionkey@0"
I20260808 00:36:59.222644 31455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "05854d26c9e843edab10157224ceed76"
format_stamp: "Formatted at 2026-08-08 00:36:59 on dist-test-slave-44d2"
server_key: "94f9d73916cc45f8029f9e9502bc731f"
server_key_iv: "a2a340d622eac1aea565aa2293c2626b"
server_key_version: "encryptionkey@0"
I20260808 00:36:59.224066 31455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:36:59.224936 31479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:36:59.225188 31455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:36:59.225327 31455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "05854d26c9e843edab10157224ceed76"
format_stamp: "Formatted at 2026-08-08 00:36:59 on dist-test-slave-44d2"
server_key: "94f9d73916cc45f8029f9e9502bc731f"
server_key_iv: "a2a340d622eac1aea565aa2293c2626b"
server_key_version: "encryptionkey@0"
I20260808 00:36:59.225406 31455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:36:59.237010 31455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:36:59.239055 31455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:36:59.239171 31455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:36:59.243279 31455 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:43201
I20260808 00:36:59.243312 31549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:43201 every 8 connection(s)
I20260808 00:36:59.243721 31455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:36:59.244755 31550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:36:59.247208 31550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76: Bootstrap starting.
I20260808 00:36:59.247818 31550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76: Neither blocks nor log segments found. Creating new log.
I20260808 00:36:59.248122 31550 log.cc:824] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76: Log is configured to *not* fsync() on all Append() calls
I20260808 00:36:59.249030 31550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76: No bootstrap required, opened a new log
I20260808 00:36:59.249029 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 31455
I20260808 00:36:59.249145 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:36:59.249375 32587 external_mini_cluster.cc:1428] Setting key bed3fd133ce66fd228b5b4bf28965935
I20260808 00:36:59.250986 31550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05854d26c9e843edab10157224ceed76" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43201 } }
I20260808 00:36:59.251160 31550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:36:59.251227 31550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05854d26c9e843edab10157224ceed76, State: Initialized, Role: FOLLOWER
I20260808 00:36:59.251415 31550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05854d26c9e843edab10157224ceed76" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43201 } }
I20260808 00:36:59.251509 31550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:36:59.251546 31550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:36:59.251595 31550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:36:59.252329 31550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05854d26c9e843edab10157224ceed76" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43201 } }
I20260808 00:36:59.252441 31550 leader_election.cc:302] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05854d26c9e843edab10157224ceed76; no voters:
I20260808 00:36:59.252638 31550 leader_election.cc:288] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:36:59.252698 31557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:36:59.252877 31557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [term 1 LEADER]: Becoming Leader. State: Replica: 05854d26c9e843edab10157224ceed76, State: Running, Role: LEADER
I20260808 00:36:59.252923 31550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:36:59.252996 31557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05854d26c9e843edab10157224ceed76" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43201 } }
I20260808 00:36:59.253402 31557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05854d26c9e843edab10157224ceed76. Latest consensus state: current_term: 1 leader_uuid: "05854d26c9e843edab10157224ceed76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05854d26c9e843edab10157224ceed76" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43201 } } }
I20260808 00:36:59.253602 31557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [sys.catalog]: This master's current role is: LEADER
I20260808 00:36:59.253533 31558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05854d26c9e843edab10157224ceed76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05854d26c9e843edab10157224ceed76" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43201 } } }
I20260808 00:36:59.254037 31558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76 [sys.catalog]: This master's current role is: LEADER
W20260808 00:36:59.255260 31579 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:36:59.255328 31579 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:36:59.255426 31565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:36:59.255637 31565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:36:59.257355 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:59.249929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54331 (local address 127.31.210.254:43201)
0808 00:36:59.250140 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:59.250149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:59.250186 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:36:59.250970 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:59.250976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:59.251025 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:59.251188 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:59.251450 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:59.253283 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:59.254862 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:59.255541 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:59.255666 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:59.256562 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:59.256567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:59.256567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:59.256795 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:59.256840 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:59.256856 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:59.256955 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:59.257077 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 00:36:59.257109 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260808 00:36:59.257365 31565 catalog_manager.cc:1347] Generated new cluster ID: 50782ecc5fff4d9ba3b201aaa1e49c8c
I20260808 00:36:59.257427 31565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:36:59.258574 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:36:59.261967 31565 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:36:59.262754 31565 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:36:59.263973 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:36:59.260356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43531 (local address 127.31.210.254:43201)
0808 00:36:59.260409 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:36:59.260411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:36:59.260580 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:36:59.260663 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:36:59.260664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:36:59.260677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:36:59.260734 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:36:59.261020 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:59.261861 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:59.262646 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:36:59.262885 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:36:59.262962 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:36:59.263701 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:36:59.263703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:36:59.263703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:36:59.263708 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:36:59.263725 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:36:59.263729 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:36:59.263788 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:36:59.263866 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 00:36:59.263877 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:36:59.267765 31565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05854d26c9e843edab10157224ceed76: Generated new TSK 0
I20260808 00:36:59.267954 31565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:36:59.270812 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 31455
W20260808 00:36:59.286280 31430 connection.cc:535] client connection to 127.31.210.254:43201 recv error: Network error: failed to read from TLS socket (remote: 127.31.210.254:43201): Connection reset by peer (error 104)
2026-08-08T00:36:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:36:59 dist-test-slave-44d2 krb5kdc[31597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:36:59 dist-test-slave-44d2 krb5kdc[31597](info): set up 2 sockets
Aug 08 00:36:59 dist-test-slave-44d2 krb5kdc[31597](info): commencing operation
krb5kdc: starting...
W20260808 00:37:00.538136 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:37:00 dist-test-slave-44d2 krb5kdc[31597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:37:00.673877 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:37517
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:37517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:00.761214 31615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:00.761440 31615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:00.761464 31615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:00.762588 31615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:00.762625 31615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:00.762639 31615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:00.762652 31615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:00.762665 31615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:00.764127 31615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:37517
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:37517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:00.764338 31615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:00.764533 31615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:00.769604 31621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:00.769611 31627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:00.769676 31622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:00.769814 31615 server_base.cc:1034] running on GCE node
I20260808 00:37:00.770095 31615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:00.770416 31615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:00.771590 31615 hybrid_clock.cc:648] HybridClock initialized: now 1786149420771572 us; error 52 us; skew 500 ppm
Aug 08 00:37:00 dist-test-slave-44d2 krb5kdc[31597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149420, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:00.774428 31615 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:37:00.774937 31615 webserver.cc:460] Webserver started at http://127.31.210.254:37781/ using document root <none> and password file <none>
I20260808 00:37:00.775192 31615 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:00.775261 31615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:00.775406 31615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:00.776587 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "d1054d22e39d491086bbf974ddfc791a"
format_stamp: "Formatted at 2026-08-08 00:37:00 on dist-test-slave-44d2"
server_key: "be042fe1fc25af5fa3652a76c81626cd"
server_key_iv: "236baa3acc440c690fcfc7bec49a3e4b"
server_key_version: "encryptionkey@0"
I20260808 00:37:00.776978 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "d1054d22e39d491086bbf974ddfc791a"
format_stamp: "Formatted at 2026-08-08 00:37:00 on dist-test-slave-44d2"
server_key: "be042fe1fc25af5fa3652a76c81626cd"
server_key_iv: "236baa3acc440c690fcfc7bec49a3e4b"
server_key_version: "encryptionkey@0"
I20260808 00:37:00.778512 31615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:37:00.779386 31636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:00.779649 31615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:00.779768 31615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "d1054d22e39d491086bbf974ddfc791a"
format_stamp: "Formatted at 2026-08-08 00:37:00 on dist-test-slave-44d2"
server_key: "be042fe1fc25af5fa3652a76c81626cd"
server_key_iv: "236baa3acc440c690fcfc7bec49a3e4b"
server_key_version: "encryptionkey@0"
I20260808 00:37:00.779850 31615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:00.783442 31615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:00.786113 31615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:00.786284 31615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:00.790773 31615 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:37517
I20260808 00:37:00.790812 31710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:37517 every 8 connection(s)
I20260808 00:37:00.791182 31615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:00.792218 31712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:00.794679 31712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a: Bootstrap starting.
I20260808 00:37:00.795301 31712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:00.795616 31712 log.cc:824] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:00.796372 31712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a: No bootstrap required, opened a new log
I20260808 00:37:00.798033 31712 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1054d22e39d491086bbf974ddfc791a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37517 } }
I20260808 00:37:00.798179 31712 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:00.798203 31712 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1054d22e39d491086bbf974ddfc791a, State: Initialized, Role: FOLLOWER
I20260808 00:37:00.798389 31712 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1054d22e39d491086bbf974ddfc791a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37517 } }
I20260808 00:37:00.798458 31712 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:00.798516 31712 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:00.798591 31712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:00.799551 31712 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1054d22e39d491086bbf974ddfc791a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37517 } }
I20260808 00:37:00.799677 31712 leader_election.cc:302] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1054d22e39d491086bbf974ddfc791a; no voters:
I20260808 00:37:00.799876 31712 leader_election.cc:288] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:00.799918 31716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:00.800098 31716 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [term 1 LEADER]: Becoming Leader. State: Replica: d1054d22e39d491086bbf974ddfc791a, State: Running, Role: LEADER
I20260808 00:37:00.800179 31712 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:00.800271 31716 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1054d22e39d491086bbf974ddfc791a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37517 } }
I20260808 00:37:00.800455 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 31615
I20260808 00:37:00.800624 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:37:00.800863 32587 external_mini_cluster.cc:1428] Setting key 942e05cbd60f8575894f005ce23c0ce7
I20260808 00:37:00.800747 31717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1054d22e39d491086bbf974ddfc791a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1054d22e39d491086bbf974ddfc791a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37517 } } }
I20260808 00:37:00.800945 31717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:00.801180 31720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:00.800731 31718 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1054d22e39d491086bbf974ddfc791a. Latest consensus state: current_term: 1 leader_uuid: "d1054d22e39d491086bbf974ddfc791a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1054d22e39d491086bbf974ddfc791a" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 37517 } } }
I20260808 00:37:00.801748 31718 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:00.801769 31720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:00.803342 31720 catalog_manager.cc:1347] Generated new cluster ID: b9eb4c1ced6947e582cf6b680dde1e88
I20260808 00:37:00.803395 31720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:00.804162 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:00.801564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38667 (local address 127.31.210.254:37517)
0808 00:37:00.801735 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:00.801747 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:00.802574 (+ 827us) server_negotiation.cc:407] Connection header received
0808 00:37:00.802809 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:00.802818 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:00.802854 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:00.803007 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:00.803693 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:00.803696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:00.803696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:00.803701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:00.803737 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:00.803751 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:00.803815 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:00.803928 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:37:00.803969 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260808 00:37:00.805104 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:00.817961 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:00.806737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60779 (local address 127.31.210.254:37517)
0808 00:37:00.806777 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:00.806779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:00.816700 (+ 9921us) server_negotiation.cc:407] Connection header received
0808 00:37:00.816801 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:00.816802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:00.816810 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:00.816869 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:00.817694 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:00.817695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:00.817696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:00.817700 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:00.817719 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:00.817724 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:00.817783 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:00.817864 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 00:37:00.817879 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:00.846500 31720 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:00.846704 31720 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:00.856964 31720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1054d22e39d491086bbf974ddfc791a: Generated new TSK 0
I20260808 00:37:00.857275 31720 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:00.860623 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 31615
2026-08-08T00:37:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:00 dist-test-slave-44d2 krb5kdc[31759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:00 dist-test-slave-44d2 krb5kdc[31759](info): set up 2 sockets
Aug 08 00:37:00 dist-test-slave-44d2 krb5kdc[31759](info): commencing operation
krb5kdc: starting...
W20260808 00:37:02.103606 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:37:02 dist-test-slave-44d2 krb5kdc[31759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:37:02.236178 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:36737
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:36737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:02.320875 31777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:02.321144 31777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:02.321187 31777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:02.322359 31777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:02.322435 31777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:02.322458 31777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:02.322479 31777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:02.322525 31777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:02.324028 31777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:36737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:36737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:02.324327 31777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:02.324594 31777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:02.329818 31782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:02.329957 31777 server_base.cc:1034] running on GCE node
W20260808 00:37:02.329838 31786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:02.329838 31784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:02.330475 31777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:02.330808 31777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:02.331984 31777 hybrid_clock.cc:648] HybridClock initialized: now 1786149422331970 us; error 40 us; skew 500 ppm
Aug 08 00:37:02 dist-test-slave-44d2 krb5kdc[31759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:02.338935 31777 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:37:02.339457 31777 webserver.cc:460] Webserver started at http://127.31.210.254:34463/ using document root <none> and password file <none>
I20260808 00:37:02.339731 31777 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:02.339804 31777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:02.339960 31777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:02.341198 31777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "9b501bdece784592b04b6f971018d931"
format_stamp: "Formatted at 2026-08-08 00:37:02 on dist-test-slave-44d2"
server_key: "5b8601326de3b9ca2dc3d71f4ef00609"
server_key_iv: "68df6e8931d124117dc47b1892b56cd1"
server_key_version: "encryptionkey@0"
I20260808 00:37:02.341714 31777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "9b501bdece784592b04b6f971018d931"
format_stamp: "Formatted at 2026-08-08 00:37:02 on dist-test-slave-44d2"
server_key: "5b8601326de3b9ca2dc3d71f4ef00609"
server_key_iv: "68df6e8931d124117dc47b1892b56cd1"
server_key_version: "encryptionkey@0"
I20260808 00:37:02.343168 31777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:02.343984 31795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:02.344225 31777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:02.344339 31777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "9b501bdece784592b04b6f971018d931"
format_stamp: "Formatted at 2026-08-08 00:37:02 on dist-test-slave-44d2"
server_key: "5b8601326de3b9ca2dc3d71f4ef00609"
server_key_iv: "68df6e8931d124117dc47b1892b56cd1"
server_key_version: "encryptionkey@0"
I20260808 00:37:02.344417 31777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:02.363667 31777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:02.366917 31777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:02.367141 31777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:02.371788 31777 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:36737
I20260808 00:37:02.371819 31867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:36737 every 8 connection(s)
I20260808 00:37:02.372236 31777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:02.372674 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 31777
I20260808 00:37:02.372833 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:37:02.373075 32587 external_mini_cluster.cc:1428] Setting key 71ac2b1847c993e007e9fd3564da2c23
I20260808 00:37:02.373476 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:02.376410 31868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931: Bootstrap starting.
I20260808 00:37:02.377101 31868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:02.377424 31868 log.cc:824] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:02.378127 31868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931: No bootstrap required, opened a new log
I20260808 00:37:02.379944 31868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b501bdece784592b04b6f971018d931" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36737 } }
I20260808 00:37:02.380120 31868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:02.380187 31868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b501bdece784592b04b6f971018d931, State: Initialized, Role: FOLLOWER
I20260808 00:37:02.380369 31868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b501bdece784592b04b6f971018d931" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36737 } }
I20260808 00:37:02.380468 31868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:02.380534 31868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:02.380587 31868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 00:37:02 dist-test-slave-44d2 krb5kdc[31759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:37:02.381199 31868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b501bdece784592b04b6f971018d931" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36737 } }
I20260808 00:37:02.381323 31868 leader_election.cc:302] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b501bdece784592b04b6f971018d931; no voters:
I20260808 00:37:02.381526 31868 leader_election.cc:288] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:02.381567 31875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:02.381690 31875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [term 1 LEADER]: Becoming Leader. State: Replica: 9b501bdece784592b04b6f971018d931, State: Running, Role: LEADER
I20260808 00:37:02.381804 31868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:02.381829 31875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b501bdece784592b04b6f971018d931" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36737 } }
I20260808 00:37:02.382283 31876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b501bdece784592b04b6f971018d931" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b501bdece784592b04b6f971018d931" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36737 } } }
I20260808 00:37:02.382381 31876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:02.382656 31878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b501bdece784592b04b6f971018d931. Latest consensus state: current_term: 1 leader_uuid: "9b501bdece784592b04b6f971018d931" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b501bdece784592b04b6f971018d931" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 36737 } } }
I20260808 00:37:02.382982 31878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:02.383176 31885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:02.383695 31885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:02.385433 31885 catalog_manager.cc:1347] Generated new cluster ID: 49fa43e8f2054fe9a7defe6034fbdcb8
I20260808 00:37:02.385485 31885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:02.385735 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:02.373696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33265 (local address 127.31.210.254:36737)
0808 00:37:02.373924 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:02.373935 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:02.373976 (+ 41us) server_negotiation.cc:407] Connection header received
0808 00:37:02.374680 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:02.374688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:02.374743 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:02.375057 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:02.375774 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:02.376937 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:02.377952 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:02.378474 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:02.378619 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:02.381396 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:02.381401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:02.381402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:02.381417 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:02.383825 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:02.384282 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:02.384286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:02.384286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:02.384401 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:02.384678 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:02.384679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:02.384679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:02.385033 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:02.385197 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:02.385342 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 00:37:02.385502 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260808 00:37:02.386760 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:02.396193 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:02.388219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44889 (local address 127.31.210.254:36737)
0808 00:37:02.388259 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:02.388260 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:02.388289 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:37:02.388344 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:02.388345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:02.388355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:02.388425 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:02.389030 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:02.390265 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:02.391251 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:02.391610 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:02.391693 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:02.393113 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:02.393114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:02.393115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:02.393124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:02.394841 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:02.395363 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:02.395364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:02.395364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:02.395432 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:02.395643 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:02.395644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:02.395645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:02.395793 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:02.395913 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:02.396001 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:37:02.396088 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:02.400161 31885 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:02.400751 31885 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:02.409096 31885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b501bdece784592b04b6f971018d931: Generated new TSK 0
I20260808 00:37:02.409350 31885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:02.412176 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 31777
2026-08-08T00:37:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:02 dist-test-slave-44d2 krb5kdc[31914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:02 dist-test-slave-44d2 krb5kdc[31914](info): set up 2 sockets
Aug 08 00:37:02 dist-test-slave-44d2 krb5kdc[31914](info): commencing operation
krb5kdc: starting...
W20260808 00:37:03.647389 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:37:03 dist-test-slave-44d2 krb5kdc[31914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:37:03.777680 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43945
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:43945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:03.866911 31927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:03.867197 31927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:03.867242 31927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:03.868508 31927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:03.868592 31927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:03.868618 31927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:03.868641 31927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:03.868669 31927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:03.870261 31927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:43945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:43945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:03.870563 31927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:03.870829 31927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:03.876015 31937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:03.876019 31940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:03.876070 31927 server_base.cc:1034] running on GCE node
W20260808 00:37:03.876127 31935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:03.876597 31927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:03.876853 31927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:03.878001 31927 hybrid_clock.cc:648] HybridClock initialized: now 1786149423877979 us; error 35 us; skew 500 ppm
Aug 08 00:37:03 dist-test-slave-44d2 krb5kdc[31914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149423, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:03.881558 31927 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:37:03.882041 31927 webserver.cc:460] Webserver started at http://127.31.210.254:37513/ using document root <none> and password file <none>
I20260808 00:37:03.882301 31927 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:03.882349 31927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:03.882477 31927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:03.883595 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "707db97d23f74725a8b9e94168a33e51"
format_stamp: "Formatted at 2026-08-08 00:37:03 on dist-test-slave-44d2"
server_key: "333cbf308b408cc517a266e8cf120468"
server_key_iv: "2344063e7b39d7b2a1d77d2b175b34a6"
server_key_version: "encryptionkey@0"
I20260808 00:37:03.884059 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "707db97d23f74725a8b9e94168a33e51"
format_stamp: "Formatted at 2026-08-08 00:37:03 on dist-test-slave-44d2"
server_key: "333cbf308b408cc517a266e8cf120468"
server_key_iv: "2344063e7b39d7b2a1d77d2b175b34a6"
server_key_version: "encryptionkey@0"
I20260808 00:37:03.885524 31927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:03.886277 31947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:03.886463 31927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:03.886569 31927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "707db97d23f74725a8b9e94168a33e51"
format_stamp: "Formatted at 2026-08-08 00:37:03 on dist-test-slave-44d2"
server_key: "333cbf308b408cc517a266e8cf120468"
server_key_iv: "2344063e7b39d7b2a1d77d2b175b34a6"
server_key_version: "encryptionkey@0"
I20260808 00:37:03.886648 31927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:03.898273 31927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:03.900714 31927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:03.900879 31927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:03.904790 31927 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:43945
I20260808 00:37:03.904824 32018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:43945 every 8 connection(s)
I20260808 00:37:03.905193 31927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:03.906057 32019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:03.908313 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51: Bootstrap starting.
I20260808 00:37:03.908962 32019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:03.909260 32019 log.cc:824] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:03.909987 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51: No bootstrap required, opened a new log
I20260808 00:37:03.911662 32019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707db97d23f74725a8b9e94168a33e51" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43945 } }
I20260808 00:37:03.911800 32019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:03.911824 32019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707db97d23f74725a8b9e94168a33e51, State: Initialized, Role: FOLLOWER
I20260808 00:37:03.911988 32019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707db97d23f74725a8b9e94168a33e51" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43945 } }
I20260808 00:37:03.912060 32019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:03.912081 32019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:03.912156 32019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:03.913038 32019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707db97d23f74725a8b9e94168a33e51" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43945 } }
I20260808 00:37:03.913153 32019 leader_election.cc:302] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707db97d23f74725a8b9e94168a33e51; no voters:
I20260808 00:37:03.913328 32019 leader_election.cc:288] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:03.913399 32022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:03.913602 32019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:03.913622 32022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [term 1 LEADER]: Becoming Leader. State: Replica: 707db97d23f74725a8b9e94168a33e51, State: Running, Role: LEADER
I20260808 00:37:03.913717 32022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707db97d23f74725a8b9e94168a33e51" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43945 } }
I20260808 00:37:03.914191 32023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "707db97d23f74725a8b9e94168a33e51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707db97d23f74725a8b9e94168a33e51" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43945 } } }
I20260808 00:37:03.914220 32024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707db97d23f74725a8b9e94168a33e51. Latest consensus state: current_term: 1 leader_uuid: "707db97d23f74725a8b9e94168a33e51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707db97d23f74725a8b9e94168a33e51" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 43945 } } }
I20260808 00:37:03.914296 32023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:03.914301 32024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:03.914602 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 31927
I20260808 00:37:03.914754 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:37:03.914887 32032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:03.914981 32587 external_mini_cluster.cc:1428] Setting key 1916951aa16aa6ef3d884cc2e5382e42
I20260808 00:37:03.915480 32032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:03.917474 32032 catalog_manager.cc:1347] Generated new cluster ID: 94e5c4d696654600a4e5e7f121a568ba
I20260808 00:37:03.917533 32032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:03.920647 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:03.915500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51639 (local address 127.31.210.254:43945)
0808 00:37:03.915795 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:03.915805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:03.915840 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:37:03.915961 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:03.915968 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:03.916035 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:03.916191 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:03.916585 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:03.917827 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:03.918630 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:03.919071 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:03.919193 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:03.919938 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:03.919942 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:03.919942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:03.920131 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:03.920168 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:03.920179 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:03.920257 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:03.920330 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 00:37:03.920396 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":102,"threads_started":1}
I20260808 00:37:03.921677 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:03.932034 32032 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:03.932135 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:03.923335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48123 (local address 127.31.210.254:43945)
0808 00:37:03.923376 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:03.923379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:03.928777 (+ 5398us) server_negotiation.cc:407] Connection header received
0808 00:37:03.928861 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:03.928862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:03.928876 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:03.928942 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:03.929208 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:03.929754 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:03.930582 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:03.930792 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:03.930873 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:03.931720 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:03.931722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:03.931722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:03.931729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:03.931749 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:03.931754 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:03.931818 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:03.931923 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:37:03.931955 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:03.932852 32032 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:03.943126 32032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 707db97d23f74725a8b9e94168a33e51: Generated new TSK 0
I20260808 00:37:03.943383 32032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:03.948525 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 31927
2026-08-08T00:37:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:04 dist-test-slave-44d2 krb5kdc[32061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:04 dist-test-slave-44d2 krb5kdc[32061](info): set up 2 sockets
Aug 08 00:37:04 dist-test-slave-44d2 krb5kdc[32061](info): commencing operation
krb5kdc: starting...
W20260808 00:37:05.189285 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:37:05 dist-test-slave-44d2 krb5kdc[32061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:37:05.320087 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:33435
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:36837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:33435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:05.404623 32078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:05.404881 32078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:05.404920 32078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:05.406066 32078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:05.406152 32078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:05.406175 32078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:05.406196 32078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:05.406227 32078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:05.407749 32078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:36837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:33435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:33435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:05.408074 32078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:05.408324 32078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:05.413880 32086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:05.414038 32088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:05.413887 32090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:05.414173 32078 server_base.cc:1034] running on GCE node
I20260808 00:37:05.414462 32078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:05.414801 32078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:05.416003 32078 hybrid_clock.cc:648] HybridClock initialized: now 1786149425415984 us; error 51 us; skew 500 ppm
Aug 08 00:37:05 dist-test-slave-44d2 krb5kdc[32061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149425, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:05.419018 32078 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:37:05.419534 32078 webserver.cc:460] Webserver started at http://127.31.210.254:33741/ using document root <none> and password file <none>
I20260808 00:37:05.419764 32078 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:05.419811 32078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:05.419914 32078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:05.421128 32078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "014a24e0e1034398b503187cfa567e64"
format_stamp: "Formatted at 2026-08-08 00:37:05 on dist-test-slave-44d2"
server_key: "ddf5bf15a1e8271f4a6cf52ab3475876"
server_key_iv: "48c46ed06f9d2f4e2cde6e56db0c3f39"
server_key_version: "encryptionkey@0"
I20260808 00:37:05.421504 32078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "014a24e0e1034398b503187cfa567e64"
format_stamp: "Formatted at 2026-08-08 00:37:05 on dist-test-slave-44d2"
server_key: "ddf5bf15a1e8271f4a6cf52ab3475876"
server_key_iv: "48c46ed06f9d2f4e2cde6e56db0c3f39"
server_key_version: "encryptionkey@0"
I20260808 00:37:05.422968 32078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:05.423776 32101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:05.424052 32078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:05.424144 32078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "014a24e0e1034398b503187cfa567e64"
format_stamp: "Formatted at 2026-08-08 00:37:05 on dist-test-slave-44d2"
server_key: "ddf5bf15a1e8271f4a6cf52ab3475876"
server_key_iv: "48c46ed06f9d2f4e2cde6e56db0c3f39"
server_key_version: "encryptionkey@0"
I20260808 00:37:05.424198 32078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:05.427510 32078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:05.429538 32078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:05.429666 32078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:05.433497 32078 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:33435
I20260808 00:37:05.433542 32176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:33435 every 8 connection(s)
I20260808 00:37:05.433887 32078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:05.434885 32177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:05.435650 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 32078
I20260808 00:37:05.435801 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:37:05.436010 32587 external_mini_cluster.cc:1428] Setting key f7df953f8bc20d356046df00996d725c
I20260808 00:37:05.438179 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64: Bootstrap starting.
I20260808 00:37:05.438669 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:05.436584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44077 (local address 127.31.210.254:33435)
0808 00:37:05.436782 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:05.436792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:05.436832 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:37:05.437321 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:05.437330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:05.437366 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:05.437524 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:05.438180 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:05.438183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:05.438184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:05.438189 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:05.438232 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:05.438247 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:05.438320 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:05.438408 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:37:05.438447 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260808 00:37:05.438894 32177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:05.439193 32177 log.cc:824] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:05.439867 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:05.440085 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64: No bootstrap required, opened a new log
I20260808 00:37:05.442471 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:05.441374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37975 (local address 127.31.210.254:33435)
0808 00:37:05.441414 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:05.441416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:05.441511 (+ 95us) server_negotiation.cc:407] Connection header received
0808 00:37:05.441645 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:05.441647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:05.441654 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:05.441716 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:05.442238 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:05.442240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:05.442240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:05.442244 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:05.442260 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:05.442264 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:05.442322 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:05.442355 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 00:37:05.442368 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:37:05.442382 32177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a24e0e1034398b503187cfa567e64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33435 } }
I20260808 00:37:05.442556 32177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:05.442590 32177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a24e0e1034398b503187cfa567e64, State: Initialized, Role: FOLLOWER
I20260808 00:37:05.442706 32177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a24e0e1034398b503187cfa567e64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33435 } }
I20260808 00:37:05.442790 32177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:05.442812 32177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:05.442883 32177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:05.443415 32177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a24e0e1034398b503187cfa567e64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33435 } }
I20260808 00:37:05.443533 32177 leader_election.cc:302] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 014a24e0e1034398b503187cfa567e64; no voters:
I20260808 00:37:05.443694 32177 leader_election.cc:288] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:05.443727 32191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:05.443879 32191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [term 1 LEADER]: Becoming Leader. State: Replica: 014a24e0e1034398b503187cfa567e64, State: Running, Role: LEADER
I20260808 00:37:05.443984 32177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:05.444007 32191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a24e0e1034398b503187cfa567e64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33435 } }
I20260808 00:37:05.444630 32193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "014a24e0e1034398b503187cfa567e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a24e0e1034398b503187cfa567e64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33435 } } }
I20260808 00:37:05.444648 32194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 014a24e0e1034398b503187cfa567e64. Latest consensus state: current_term: 1 leader_uuid: "014a24e0e1034398b503187cfa567e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a24e0e1034398b503187cfa567e64" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 33435 } } }
I20260808 00:37:05.444757 32193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:05.444782 32194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:05.445104 32202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:05.445708 32202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:05.447489 32202 catalog_manager.cc:1347] Generated new cluster ID: 06a4f912065e4b86afd141d6e241d4d5
I20260808 00:37:05.447589 32202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:05.463169 32202 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:05.463390 32202 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:05.467814 32202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 014a24e0e1034398b503187cfa567e64: Generated new TSK 0
I20260808 00:37:05.468072 32202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:05.472632 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 32078
2026-08-08T00:37:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1530 ms)
[----------] 18 tests from AuthTokenIssuingTest (28303 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:05 dist-test-slave-44d2 krb5kdc[32217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:05 dist-test-slave-44d2 krb5kdc[32217](info): set up 2 sockets
Aug 08 00:37:05 dist-test-slave-44d2 krb5kdc[32217](info): commencing operation
krb5kdc: starting...
W20260808 00:37:06.773037 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:37:06 dist-test-slave-44d2 krb5kdc[32217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:37:06.906805 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:42271
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:42271,127.31.210.253:36053,127.31.210.252:41281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:06.992568 32237 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:37:06.992883 32237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:06.992918 32237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:06.992965 32237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:06.994221 32237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:06.994297 32237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:06.994320 32237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:06.994340 32237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:06.994369 32237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:06.995916 32237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:42271,127.31.210.253:36053,127.31.210.252:41281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:06.996238 32237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:06.996508 32237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:07.001611 32244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:07.001667 32245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:07.001664 32237 server_base.cc:1034] running on GCE node
W20260808 00:37:07.001611 32248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:07.002230 32237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:07.002599 32237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:07.003790 32237 hybrid_clock.cc:648] HybridClock initialized: now 1786149427003773 us; error 44 us; skew 500 ppm
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:07.006438 32237 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:37:07.006850 32237 webserver.cc:460] Webserver started at http://127.31.210.254:32969/ using document root <none> and password file <none>
I20260808 00:37:07.007040 32237 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:07.007083 32237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:07.007174 32237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:07.008253 32237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "d8a7bc8e549c40b99465372ac66e0411"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "2b2de6546f0e75aa843f435948ce79e8"
server_key_iv: "28b69f7a8adf31f0e25f5e98e5fe4a83"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.008661 32237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "d8a7bc8e549c40b99465372ac66e0411"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "2b2de6546f0e75aa843f435948ce79e8"
server_key_iv: "28b69f7a8adf31f0e25f5e98e5fe4a83"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.009971 32237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:07.010679 32256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:07.010874 32237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:07.010962 32237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "d8a7bc8e549c40b99465372ac66e0411"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "2b2de6546f0e75aa843f435948ce79e8"
server_key_iv: "28b69f7a8adf31f0e25f5e98e5fe4a83"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.011016 32237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:07.025513 32237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:07.027588 32237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:07.027719 32237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:07.032075 32237 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:42271
I20260808 00:37:07.032109 32329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:42271 every 8 connection(s)
I20260808 00:37:07.032734 32237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:07.033141 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 32237
I20260808 00:37:07.033260 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:37:07.033478 32587 external_mini_cluster.cc:1428] Setting key 0107cc7e45245f80ae15697362e453c2
I20260808 00:37:07.033658 32331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:07.036096 32331 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.210.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.253@KRBTEST.COM" created.
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:37:07.049312 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.036992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:42271 (local address 127.0.0.1:60528)
0808 00:37:07.037212 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.037255 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.037772 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.038070 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.038073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.038429 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.038605 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.038609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.039689 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.039690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.040408 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.040410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.040930 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.040931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.041005 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.041572 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.041581 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.045702 (+ 4121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.047679 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.047687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.047689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.047890 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.048152 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.048153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.048153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.048248 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.048706 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.048712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.048779 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.048889 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.049132 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":99,"thread_start_us":108,"threads_started":1}
I20260808 00:37:07.049314 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.037013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60528 (local address 127.31.210.254:42271)
0808 00:37:07.037239 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.037249 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.037366 (+ 117us) server_negotiation.cc:407] Connection header received
0808 00:37:07.037949 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.037951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.037968 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.038064 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.038718 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.039581 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.040515 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.040831 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.040949 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.045797 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.045806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.045806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.045814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.047574 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.047989 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.047992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.047993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.048082 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.048317 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.048318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.048318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.048620 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.048715 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.048923 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.049056 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":128,"threads_started":1}
I20260808 00:37:07.050060 32331 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:07.050444 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.050340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36198)
0808 00:37:07.050384 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.050427 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 00:37:07.050598 32258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
I20260808 00:37:07.051018 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.050920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36204)
0808 00:37:07.050965 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.050990 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:37:07.051127 32331 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:36053: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab.
Entry for principal kudu/127.31.210.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab.
I20260808 00:37:07.063865 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:36053
--webserver_interface=127.31.210.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab
--principal=kudu/127.31.210.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:42271,127.31.210.253:36053,127.31.210.252:41281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:37:07.100373 32331 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } attempt: 1
I20260808 00:37:07.100957 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.100819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36210)
0808 00:37:07.100884 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.100925 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:37:07.101326 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.101226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36212)
0808 00:37:07.101274 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.101306 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:37:07.101426 32331 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:36053: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
W20260808 00:37:07.148712 32340 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:37:07.148959 32340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:07.148993 32340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:07.149035 32340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:07.150210 32340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:07.150286 32340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:07.150310 32340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:07.150331 32340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:07.150362 32340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:07.151849 32340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:42271,127.31.210.253:36053,127.31.210.252:41281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab
--principal=kudu/127.31.210.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:36053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.210.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:07.152161 32340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:07.152406 32340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:07.158437 32352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:07.158440 32349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:07.158442 32350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:07.158601 32340 server_base.cc:1034] running on GCE node
I20260808 00:37:07.158995 32340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:07.159350 32340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:07.160542 32340 hybrid_clock.cc:648] HybridClock initialized: now 1786149427160529 us; error 27 us; skew 500 ppm
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:07.163105 32340 init.cc:377] Logged in from keytab as kudu/127.31.210.253@KRBTEST.COM (short username kudu)
I20260808 00:37:07.163609 32340 webserver.cc:460] Webserver started at http://127.31.210.253:44757/ using document root <none> and password file <none>
I20260808 00:37:07.163868 32340 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:07.163941 32340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:07.164100 32340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:07.165309 32340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data/instance:
uuid: "eecefcafda94436aa629969286be5c83"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "1f035b93f5f4f00b8c472ed5db430a84"
server_key_iv: "3de298492ca37b7d2ef1bd86ae96f92f"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.165678 32340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/wal/instance:
uuid: "eecefcafda94436aa629969286be5c83"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "1f035b93f5f4f00b8c472ed5db430a84"
server_key_iv: "3de298492ca37b7d2ef1bd86ae96f92f"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.167197 32340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:07.168043 32363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:07.168345 32340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:07.168457 32340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/wal
uuid: "eecefcafda94436aa629969286be5c83"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "1f035b93f5f4f00b8c472ed5db430a84"
server_key_iv: "3de298492ca37b7d2ef1bd86ae96f92f"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.168584 32340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:07.169572 32331 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } attempt: 2
I20260808 00:37:07.170050 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.169910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36226)
0808 00:37:07.169977 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.170017 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 00:37:07.170444 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.170314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36238)
0808 00:37:07.170348 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.170405 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 00:37:07.170598 32331 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:36053: Network error: Client connection negotiation failed: client connection to 127.31.210.253:36053: connect: Connection refused (error 111)
I20260808 00:37:07.199661 32340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:07.201829 32340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:07.201958 32340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:07.206316 32340 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.253:36053
I20260808 00:37:07.206353 32430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.253:36053 every 8 connection(s)
I20260808 00:37:07.206707 32340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
I20260808 00:37:07.207602 32431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:07.209271 32431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:07.210201 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 32340
I20260808 00:37:07.210314 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-1/wal/instance
I20260808 00:37:07.210503 32587 external_mini_cluster.cc:1428] Setting key 352971b9dfdeda21a66d04fff16920ae
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.210.252@KRBTEST.COM; defaulting to no policy
I20260808 00:37:07.220348 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.209990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.31.210.254:42271)
0808 00:37:07.210066 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.210069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.210262 (+ 193us) server_negotiation.cc:407] Connection header received
0808 00:37:07.210842 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.210844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.210860 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.210933 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.211540 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.212293 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.213844 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.214116 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.214259 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.216927 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.216929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.216930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.216938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.218558 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.219031 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.219033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.219033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.219102 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.219417 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.219418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.219418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.219706 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.219819 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.220128 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.220252 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:37:07.220484 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.209958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:42271 (local address 127.0.0.1:60540)
0808 00:37:07.210128 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.210171 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.210739 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.211004 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.211007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.211257 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.211449 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.211453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.212399 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.212401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.213754 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.213757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.214264 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.214265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.214416 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.214992 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.215003 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.216809 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.218674 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.218683 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.218686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.218945 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.219187 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.219188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.219188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.219326 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.219809 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.219817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.219967 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.220082 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.220256 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260808 00:37:07.221103 32431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.252@KRBTEST.COM" created.
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:37:07.228588 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.221362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36242)
0808 00:37:07.221414 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.221425 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.221463 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.221854 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.221855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.222022 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.222085 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.222087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.223029 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.223030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.223524 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.223525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.224004 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.224005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.224106 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.224528 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.224533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.225669 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.227446 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.227447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.227447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.227621 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.227882 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.227882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.227883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.227945 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.228241 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.228242 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.228280 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.228347 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.228451 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 00:37:07.228762 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.221444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.31.210.253:36053)
0808 00:37:07.221684 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.221688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.221712 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:37:07.221734 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.221736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.221749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.221832 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.222164 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.222934 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.223611 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.223913 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.224008 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.225765 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.225773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.225774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.225780 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.227343 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.227723 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.227726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.227726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.227783 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.228018 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.228019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.228019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.228165 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.228251 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.228389 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.228526 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":194,"thread_start_us":122,"threads_started":1}
I20260808 00:37:07.229429 32431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:07.229774 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.229653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46328)
0808 00:37:07.229702 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.229754 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 00:37:07.229907 32366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
I20260808 00:37:07.230315 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.230209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46340)
0808 00:37:07.230262 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.230286 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:37:07.230414 32431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41281: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab.
Entry for principal kudu/127.31.210.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab.
I20260808 00:37:07.240391 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:41281
--webserver_interface=127.31.210.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab
--principal=kudu/127.31.210.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:42271,127.31.210.253:36053,127.31.210.252:41281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:37:07.261765 32331 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } attempt: 3
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:37:07.269603 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.262313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36244)
0808 00:37:07.262386 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.262400 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.262462 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.262658 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.262659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.262848 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.262921 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.262923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.263687 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.263688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.264230 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.264232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.264704 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.264705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.264796 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.265310 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.265316 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.266776 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.268502 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.268503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.268504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.268699 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.268919 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.268920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.268921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.268988 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.269256 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.269257 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.269304 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.269388 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.269495 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 00:37:07.269603 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.262386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.31.210.253:36053)
0808 00:37:07.262441 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.262444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.262457 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:37:07.262551 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.262552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.262567 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.262647 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.263041 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.263577 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.264322 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.264621 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.264706 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.266880 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.266882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.266882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.266891 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.268379 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.268787 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.268789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.268789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.268850 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.269060 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.269061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.269062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.269180 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.269251 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.269413 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.269505 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:37:07.270426 32331 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:07.270810 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.270717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46348)
0808 00:37:07.270754 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.270789 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 00:37:07.271202 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.271101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46354)
0808 00:37:07.271152 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.271180 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:37:07.271315 32331 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41281: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
I20260808 00:37:07.279640 32431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } attempt: 1
I20260808 00:37:07.280174 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.279990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46362)
0808 00:37:07.280085 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.280135 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:37:07.280704 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.280592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46364)
0808 00:37:07.280639 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.280671 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:37:07.280839 32431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41281: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
I20260808 00:37:07.302489 32331 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } attempt: 1
I20260808 00:37:07.303136 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.302892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46380)
0808 00:37:07.303043 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.303095 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260808 00:37:07.303576 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.303463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46384)
0808 00:37:07.303520 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.303552 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 00:37:07.303686 32331 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41281: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
W20260808 00:37:07.329721 32438 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:37:07.329983 32438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:07.330015 32438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:07.330068 32438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:07.331233 32438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:07.331303 32438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:07.331327 32438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:07.331350 32438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:07.331379 32438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:07.332901 32438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:42271,127.31.210.253:36053,127.31.210.252:41281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab
--principal=kudu/127.31.210.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:41281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.210.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:07.333194 32438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:07.333452 32438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:37:07.338567 32438 server_base.cc:1034] running on GCE node
W20260808 00:37:07.338480 32449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:07.338548 32450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:07.338474 32453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:07.339003 32438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:07.339318 32438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:07.340562 32438 hybrid_clock.cc:648] HybridClock initialized: now 1786149427340552 us; error 38 us; skew 500 ppm
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:07.348647 32438 init.cc:377] Logged in from keytab as kudu/127.31.210.252@KRBTEST.COM (short username kudu)
I20260808 00:37:07.349028 32431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } attempt: 2
I20260808 00:37:07.349210 32438 webserver.cc:460] Webserver started at http://127.31.210.252:42771/ using document root <none> and password file <none>
I20260808 00:37:07.349457 32438 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:07.349485 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.349356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46396)
0808 00:37:07.349420 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.349457 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:37:07.349526 32438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:07.349680 32438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:07.349800 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.349704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46412)
0808 00:37:07.349749 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.349781 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:37:07.349893 32431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:41281: Network error: Client connection negotiation failed: client connection to 127.31.210.252:41281: connect: Connection refused (error 111)
I20260808 00:37:07.350875 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data/instance:
uuid: "dbc9d5574bcb49d9a5814b033b54c5de"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "e6c35aad3fb9d20d116652006c1da134"
server_key_iv: "ee671fd8d2ac43bb563d0b7c1106dadc"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.351277 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/wal/instance:
uuid: "dbc9d5574bcb49d9a5814b033b54c5de"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "e6c35aad3fb9d20d116652006c1da134"
server_key_iv: "ee671fd8d2ac43bb563d0b7c1106dadc"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.352758 32438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:07.353536 32463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:07.353753 32438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:07.353868 32438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/wal
uuid: "dbc9d5574bcb49d9a5814b033b54c5de"
format_stamp: "Formatted at 2026-08-08 00:37:07 on dist-test-slave-44d2"
server_key: "e6c35aad3fb9d20d116652006c1da134"
server_key_iv: "ee671fd8d2ac43bb563d0b7c1106dadc"
server_key_version: "encryptionkey@0"
I20260808 00:37:07.353945 32438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:07.362932 32438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:07.365558 32438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:07.365732 32438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:07.369859 32438 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.252:41281
I20260808 00:37:07.369897 32530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.252:41281 every 8 connection(s)
I20260808 00:37:07.370275 32438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
I20260808 00:37:07.371186 32531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:07.372958 32531 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:07.376915 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 32438
I20260808 00:37:07.377038 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149138237870-32587-0/minicluster-data/master-2/wal/instance
I20260808 00:37:07.377278 32587 external_mini_cluster.cc:1428] Setting key cce970871593f8273b4c782a46378b1e
I20260808 00:37:07.378849 32331 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } attempt: 2
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:37:07.383442 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.373716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.31.210.254:42271)
0808 00:37:07.373781 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.373784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.373942 (+ 158us) server_negotiation.cc:407] Connection header received
0808 00:37:07.374404 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.374405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.374414 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.374477 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.375122 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.375602 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.376620 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.376801 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.376857 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.379984 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.379986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.379987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.380002 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.381739 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.382280 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.382281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.382282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.382342 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.382672 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.382673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.382673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.382834 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.382954 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.383302 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.383378 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:37:07.383944 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.373645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:42271 (local address 127.0.0.1:60554)
0808 00:37:07.373811 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.373855 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.374319 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.374575 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.374579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.374823 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.375025 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.375029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.375683 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.375684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.376452 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.376453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.376893 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.376894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.377062 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.377970 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.377982 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.379898 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.381824 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.381832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.381835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.382164 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.382432 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.382433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.382433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.382587 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.382946 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.382954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.383128 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.383259 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.383412 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:37:07.384584 32531 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:07.387866 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.379116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46432)
0808 00:37:07.379169 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.379179 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.379215 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.379450 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.379451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.379621 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.379659 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.379661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.380511 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.380513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.381284 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.381287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.381816 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.381818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.381881 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.382315 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.382321 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.384403 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.386676 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.386677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.386677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.386890 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.387153 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.387154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.387154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.387223 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.387539 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.387540 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.387590 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.387667 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.387760 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:37:07.388263 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.379179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46432 (local address 127.31.210.252:41281)
0808 00:37:07.379310 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.379312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.379319 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:07.379338 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.379338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.379347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.379421 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.379744 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.380387 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.381369 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.381721 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.381802 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.384501 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.384508 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.384509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.384516 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.386590 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.386995 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.386998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.386999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.387072 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.387294 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.387295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.387295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.387457 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.387591 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.388030 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.388150 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:37:07.390079 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.377755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.31.210.252:41281)
0808 00:37:07.377917 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.377921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.377996 (+ 75us) server_negotiation.cc:407] Connection header received
0808 00:37:07.378051 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.378053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.378072 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.378135 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.378913 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.379965 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.381068 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.381520 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.381606 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.385331 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.385333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.385333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.385340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.388467 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.388883 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.388884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.388885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.388939 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.389176 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.389177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.389177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.389298 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.389794 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.389834 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.389907 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260808 00:37:07.390528 32331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411: Bootstrap starting.
I20260808 00:37:07.391185 32331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:07.391522 32331 log.cc:824] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:07.392436 32331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411: No bootstrap required, opened a new log
I20260808 00:37:07.392978 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.384845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:36053 (local address 127.0.0.1:36248)
0808 00:37:07.384886 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.384895 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.384929 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.385110 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.385112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.385286 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.385341 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.385343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.386298 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.386299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.386907 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.386909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.387445 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.387446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.387526 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.387935 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.387941 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.389645 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.391462 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.391464 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.391464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.391683 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.392246 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.392247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.392247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.392329 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.392648 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.392649 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.392699 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.392784 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.392873 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:37:07.392988 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.384933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.31.210.253:36053)
0808 00:37:07.384976 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.384978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.384987 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:07.385013 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.385014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.385024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.385089 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.385755 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.386201 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.387023 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.387375 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.387427 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.389749 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.389750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.389750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.389758 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.391361 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.391760 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.391761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.391761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.391841 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.392389 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.392390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.392390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.392558 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.392651 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.392814 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.392895 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:37:07.393342 32531 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:07.394542 32331 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.394680 32331 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a7bc8e549c40b99465372ac66e0411, State: Initialized, Role: FOLLOWER
I20260808 00:37:07.394767 32331 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.395066 32542 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.395226 32542 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:07.395131 32331 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
W20260808 00:37:07.398598 32556 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:07.398655 32556 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:07.400959 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.393542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46436)
0808 00:37:07.393592 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.393602 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.393645 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.393759 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.393759 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.393902 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.393940 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.393941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.394790 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.394791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.395481 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.395483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.395984 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.395984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.396022 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.396405 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.396409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.397893 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.399928 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.399929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.399929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.400153 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.400341 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.400341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.400342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.400390 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.400685 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.400686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.400724 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.400772 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.400865 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:37:07.401286 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.393617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.31.210.252:41281)
0808 00:37:07.393644 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.393645 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.393653 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:07.393697 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.393697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.393707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.393757 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.394023 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.394719 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.395564 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.395903 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.395987 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.397984 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.397985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.397985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.397991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.399837 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.400227 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.400228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.400228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.400277 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.400450 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.400451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.400451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.400612 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.401087 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.401132 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.401202 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260808 00:37:07.402096 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.392100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60564 (local address 127.31.210.254:42271)
0808 00:37:07.392138 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.392140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.392181 (+ 41us) server_negotiation.cc:407] Connection header received
0808 00:37:07.392229 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.392230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.392239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.392301 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.393005 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.393670 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.394621 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.394895 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.394949 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.397929 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.397930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.397930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.397938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.400194 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.400620 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.400622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.400622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.400679 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.400878 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.400879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.400879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.400998 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.401932 (+ 934us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.401981 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.402042 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:07.399820 32237 master_runner.cc:186] Error getting master registration for 127.31.210.253:36053: OK, instance_id { permanent_uuid: "eecefcafda94436aa629969286be5c83" instance_seqno: 1786149427205013 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:37:07.402794 32237 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:37:07.402774 32531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: Bootstrap starting.
I20260808 00:37:07.403527 32531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:07.403818 32531 log.cc:824] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:07.404701 32531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: No bootstrap required, opened a new log
I20260808 00:37:07.406714 32531 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.406889 32531 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc9d5574bcb49d9a5814b033b54c5de, State: Initialized, Role: FOLLOWER
I20260808 00:37:07.406980 32531 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.407357 32531 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:07.407334 32560 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.407588 32560 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
W20260808 00:37:07.410008 32575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:07.410125 32575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:07.412626 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.403393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.31.210.253:36053)
0808 00:37:07.403438 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.403440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.403448 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:07.403471 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.403472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.403481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.403532 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.404081 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.404763 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.405715 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.406029 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.406124 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.409279 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.409281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.409282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.409290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.411204 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.411601 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.411602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.411603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.411659 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.411855 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.411856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.411856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.412005 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.412173 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.412391 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.412511 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:07.410923 32438 master_runner.cc:186] Error getting master registration for 127.31.210.253:36053: OK, instance_id { permanent_uuid: "eecefcafda94436aa629969286be5c83" instance_seqno: 1786149427205013 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:37:07.413609 32438 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:37:07.414708 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:07.415659 32291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d8a7bc8e549c40b99465372ac66e0411"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60564
I20260808 00:37:07.415804 32291 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 00:37:07.415860 32291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:07.416702 32291 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.416973 32291 leader_election.cc:288] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [CANDIDATE]: Term 1 election: Requested vote from peers eecefcafda94436aa629969286be5c83 (127.31.210.253:36053), dbc9d5574bcb49d9a5814b033b54c5de (127.31.210.252:41281)
I20260808 00:37:07.417256 32498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8a7bc8e549c40b99465372ac66e0411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dbc9d5574bcb49d9a5814b033b54c5de"
I20260808 00:37:07.417388 32498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:07.418390 32498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8a7bc8e549c40b99465372ac66e0411 in term 1.
I20260808 00:37:07.418591 32258 leader_election.cc:302] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8a7bc8e549c40b99465372ac66e0411, dbc9d5574bcb49d9a5814b033b54c5de; no voters:
I20260808 00:37:07.418749 32542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:07.418924 32542 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [term 1 LEADER]: Becoming Leader. State: Replica: d8a7bc8e549c40b99465372ac66e0411, State: Running, Role: LEADER
I20260808 00:37:07.419025 32542 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.419487 32542 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8a7bc8e549c40b99465372ac66e0411. Latest consensus state: current_term: 1 leader_uuid: "d8a7bc8e549c40b99465372ac66e0411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.419575 32542 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:07.426096 32586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:07.426324 32586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:07.436908 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.424809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.31.210.253:36053)
0808 00:37:07.424871 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.424874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.425552 (+ 678us) server_negotiation.cc:407] Connection header received
0808 00:37:07.425695 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.425696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.425720 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.425752 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.427223 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.428011 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.429694 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.430017 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.430060 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.432406 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.432408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.432408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.432429 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.434242 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.435050 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.435052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.435053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.435131 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.435762 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.435776 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.435777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.435930 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.436026 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.436706 (+ 680us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.436795 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:07.440292 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.425266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.31.210.254:42271)
0808 00:37:07.425318 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.425321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.428184 (+ 2863us) server_negotiation.cc:407] Connection header received
0808 00:37:07.428282 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.428283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.428292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.428322 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.430793 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.431517 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.434684 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.435008 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.435050 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.436582 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.436584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.436584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.436594 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.438401 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.439053 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.439054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.439055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.439118 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.439350 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.439352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.439352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.439755 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.439880 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.440092 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.440173 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:37:07.441035 32431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } attempt: 3
I20260808 00:37:07.442480 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.425059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.31.210.252:41281)
0808 00:37:07.425121 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.425123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.427274 (+ 2151us) server_negotiation.cc:407] Connection header received
0808 00:37:07.427367 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.427368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.427378 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.427408 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.429133 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.429843 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.432569 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.432894 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.432942 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.438574 (+ 5632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.438576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.438576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.438585 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.440351 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.441360 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.441362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.441362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.441426 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.441744 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.441757 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.441758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.441900 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.441986 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.442306 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.442386 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 00:37:07 dist-test-slave-44d2 krb5kdc[32217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149427, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:37:07.447435 32498 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [term 1 FOLLOWER]: Refusing update from remote peer d8a7bc8e549c40b99465372ac66e0411: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:07.447830 32582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:07.449275 32560 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8a7bc8e549c40b99465372ac66e0411. Latest consensus state: current_term: 1 leader_uuid: "d8a7bc8e549c40b99465372ac66e0411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.449373 32560 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:07.450937 32433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.441292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:41281 (local address 127.0.0.1:46468)
0808 00:37:07.441378 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:07.441388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:07.441424 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:07.441716 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:07.441717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:07.441876 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:07.441923 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.441925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.442894 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.442896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.443713 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:07.443716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.444242 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.444244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:07.444300 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.444724 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:07.444729 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:07.446207 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:07.449642 (+ 3435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.449643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.449644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.449879 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.450115 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:07.450116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:07.450117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:07.450208 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:07.450569 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:07.450570 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:07.450635 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 00:37:07.450703 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 00:37:07.450820 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 00:37:07.451651 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:07.441535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46468 (local address 127.31.210.252:41281)
0808 00:37:07.441578 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.441580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.441587 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:07.441605 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.441606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.441617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.441669 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.442002 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.442762 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.443807 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:07.444134 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:07.444192 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:07.446292 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:07.446294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:07.446294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:07.446302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:07.448172 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.449950 (+ 1778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.449952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.449952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.450024 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:07.450294 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:07.450295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:07.450296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:07.450460 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:07.450527 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:07.451458 (+ 931us) server_negotiation.cc:299] Negotiation successful
0808 00:37:07.451555 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:37:07.451740 32582 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8a7bc8e549c40b99465372ac66e0411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.451835 32582 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:07.452459 32605 mvcc.cc:205] Tried to move back new op lower bound from 7316068054821072896 to 7316068054709186560. Current Snapshot: MvccSnapshot[applied={T|T < 7316068054821072896}]
I20260808 00:37:07.453619 32431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: Bootstrap starting.
I20260808 00:37:07.454402 32431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:07.454739 32431 log.cc:824] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:07.455014 32560 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8a7bc8e549c40b99465372ac66e0411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.455124 32560 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:07.455992 32431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: No bootstrap required, opened a new log
I20260808 00:37:07.458204 32431 raft_consensus.cc:348] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.458350 32431 raft_consensus.cc:729] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecefcafda94436aa629969286be5c83, State: Initialized, Role: FOLLOWER
I20260808 00:37:07.458436 32431 consensus_queue.cc:260] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } }
I20260808 00:37:07.458961 32431 sys_catalog.cc:564] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:07.459008 32401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8a7bc8e549c40b99465372ac66e0411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eecefcafda94436aa629969286be5c83"
I20260808 00:37:07.459107 32401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:07.460062 32401 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8a7bc8e549c40b99465372ac66e0411 in term 1.
I20260808 00:37:07.460263 32398 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [term 1 FOLLOWER]: Refusing update from remote peer d8a7bc8e549c40b99465372ac66e0411: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:07.458923 32612 sys_catalog.cc:455] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.460660 32612 sys_catalog.cc:458] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:07.458609 32603 mvcc.cc:205] Tried to move back new op lower bound from 7316068054821072896 to 7316068054709186560. Current Snapshot: MvccSnapshot[applied={T|T < 7316068054821072896}]
I20260808 00:37:07.460777 32542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:07.461640 32586 catalog_manager.cc:1347] Generated new cluster ID: 9060beb3eba7430ca3e63c5c88d6b74e
I20260808 00:37:07.461691 32586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:07.462263 32612 sys_catalog.cc:455] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8a7bc8e549c40b99465372ac66e0411. Latest consensus state: current_term: 1 leader_uuid: "d8a7bc8e549c40b99465372ac66e0411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.462698 32612 sys_catalog.cc:458] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:07.462793 32612 sys_catalog.cc:455] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8a7bc8e549c40b99465372ac66e0411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.462908 32612 sys_catalog.cc:458] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:07.464713 32629 mvcc.cc:205] Tried to move back new op lower bound from 7316068054821072896 to 7316068054709186560. Current Snapshot: MvccSnapshot[applied={T|T < 7316068054821072896}]
W20260808 00:37:07.465387 32636 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:07.465478 32636 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:07.477640 32542 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8a7bc8e549c40b99465372ac66e0411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8a7bc8e549c40b99465372ac66e0411" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 42271 } } peers { permanent_uuid: "eecefcafda94436aa629969286be5c83" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 36053 } } peers { permanent_uuid: "dbc9d5574bcb49d9a5814b033b54c5de" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 41281 } } }
I20260808 00:37:07.477770 32542 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:07.481244 32586 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:07.481998 32586 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:07.490746 32586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8a7bc8e549c40b99465372ac66e0411: Generated new TSK 0
I20260808 00:37:07.490973 32586 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:37:07.513873 32337 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.511997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.31.210.254:42271)
0808 00:37:07.512045 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.512060 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.513145 (+ 1085us) server_negotiation.cc:407] Connection header received
0808 00:37:07.513284 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.513285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.513297 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.513331 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.513801 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60578: BlockingRecv error: recv got EOF from 127.0.0.1:60578 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 00:37:07.514729 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.512457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.31.210.253:36053)
0808 00:37:07.512528 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.512530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.514151 (+ 1621us) server_negotiation.cc:407] Connection header received
0808 00:37:07.514253 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.514254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.514265 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.514297 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.514663 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36278: BlockingRecv error: recv got EOF from 127.0.0.1:36278 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 00:37:07.515563 32540 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.512819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.31.210.252:41281)
0808 00:37:07.512860 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.512862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.514954 (+ 2092us) server_negotiation.cc:407] Connection header received
0808 00:37:07.515139 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.515140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.515152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.515183 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.515500 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46484: BlockingRecv error: recv got EOF from 127.0.0.1:46484 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 00:37:07.522176 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.521278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.31.210.253:36053)
0808 00:37:07.521318 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.521320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.521572 (+ 252us) server_negotiation.cc:407] Connection header received
0808 00:37:07.521691 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.521692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.521713 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.521745 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.522123 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36290: BlockingRecv error: recv got EOF from 127.0.0.1:36290 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 00:37:07.525475 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.524620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.31.210.253:36053)
0808 00:37:07.524657 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.524659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.524909 (+ 250us) server_negotiation.cc:407] Connection header received
0808 00:37:07.525016 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.525017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.525029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.525073 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.525423 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36306: BlockingRecv error: recv got EOF from 127.0.0.1:36306 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:37:07.529758 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.528890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.31.210.253:36053)
0808 00:37:07.528930 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.528932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.529195 (+ 263us) server_negotiation.cc:407] Connection header received
0808 00:37:07.529300 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.529301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.529313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.529344 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.529706 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36308: BlockingRecv error: recv got EOF from 127.0.0.1:36308 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:37:07.535952 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.535081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.31.210.253:36053)
0808 00:37:07.535122 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.535124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.535390 (+ 266us) server_negotiation.cc:407] Connection header received
0808 00:37:07.535497 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.535498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.535510 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.535540 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.535896 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36322: BlockingRecv error: recv got EOF from 127.0.0.1:36322 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 00:37:07.546532 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.545602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.31.210.253:36053)
0808 00:37:07.545645 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.545647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.545904 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:37:07.546011 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.546013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.546026 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.546078 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.546466 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36334: BlockingRecv error: recv got EOF from 127.0.0.1:36334 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 00:37:07.565146 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.563999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.31.210.253:36053)
0808 00:37:07.564044 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.564060 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.564376 (+ 316us) server_negotiation.cc:407] Connection header received
0808 00:37:07.564547 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.564549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.564564 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.564602 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.565082 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36342: BlockingRecv error: recv got EOF from 127.0.0.1:36342 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 00:37:07.599562 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.598618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36348 (local address 127.31.210.253:36053)
0808 00:37:07.598668 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.598671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.598830 (+ 159us) server_negotiation.cc:407] Connection header received
0808 00:37:07.598954 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.598955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.598975 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.599072 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.599498 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36348: BlockingRecv error: recv got EOF from 127.0.0.1:36348 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:37:07.665550 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.664758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.31.210.253:36053)
0808 00:37:07.664834 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.664837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.664977 (+ 140us) server_negotiation.cc:407] Connection header received
0808 00:37:07.665066 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.665067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.665080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.665124 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.665487 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36350: BlockingRecv error: recv got EOF from 127.0.0.1:36350 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 00:37:07.795737 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:07.794984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.31.210.253:36053)
0808 00:37:07.795064 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:07.795066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:07.795201 (+ 135us) server_negotiation.cc:407] Connection header received
0808 00:37:07.795243 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:07.795249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:07.795261 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:07.795316 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:07.795674 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36352: BlockingRecv error: recv got EOF from 127.0.0.1:36352 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 00:37:08.053943 32435 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:08.053152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.31.210.253:36053)
0808 00:37:08.053227 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:08.053229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:08.053240 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:37:08.053293 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:08.053294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:08.053309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:08.053412 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:08.053865 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36362: BlockingRecv error: recv got EOF from 127.0.0.1:36362 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:37:08.410630 32575 catalog_manager.cc:1259] Loaded cluster ID: 9060beb3eba7430ca3e63c5c88d6b74e
I20260808 00:37:08.410713 32575 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: loading cluster ID for follower catalog manager: success
I20260808 00:37:08.411527 32575 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: acquiring CA information for follower catalog manager: success
I20260808 00:37:08.411809 32575 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dbc9d5574bcb49d9a5814b033b54c5de: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:37:08.466073 32636 catalog_manager.cc:1259] Loaded cluster ID: 9060beb3eba7430ca3e63c5c88d6b74e
I20260808 00:37:08.466167 32636 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: loading cluster ID for follower catalog manager: success
I20260808 00:37:08.467144 32636 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: acquiring CA information for follower catalog manager: success
I20260808 00:37:08.467522 32636 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eecefcafda94436aa629969286be5c83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:37:08.570230 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:08.567573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.31.210.253:36053)
0808 00:37:08.567711 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:08.567713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:08.567722 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:08.567756 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:08.567757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:08.567810 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:08.567901 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:08.568074 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.568668 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.569518 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.569716 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.569773 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:08.569931 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:08.570042 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:08.570100 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:08.570145 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 00:37:08.570163 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260808 00:37:08.574723 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:08.572286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.31.210.252:41281)
0808 00:37:08.572422 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:08.572425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:08.572435 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:08.572474 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:08.572475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:08.572580 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:08.572666 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:08.572799 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.573271 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.573997 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.574196 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.574251 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:08.574396 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:08.574497 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:08.574552 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:08.574589 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 00:37:08.574607 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260808 00:37:08.579538 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:08.576507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.31.210.253:36053)
0808 00:37:08.576550 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:08.576552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:08.576754 (+ 202us) server_negotiation.cc:407] Connection header received
0808 00:37:08.576879 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:08.576880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:08.576891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:08.576947 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:08.577102 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.577704 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.578705 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.579012 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.579079 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:08.579239 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:08.579273 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:08.579331 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:08.579407 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 00:37:08.579435 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:08.579782 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:08.576875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46502 (local address 127.31.210.252:41281)
0808 00:37:08.576939 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:08.576941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:08.577113 (+ 172us) server_negotiation.cc:407] Connection header received
0808 00:37:08.577212 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:08.577213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:08.577225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:08.577279 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:08.577415 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.578105 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.578977 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.579276 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.579336 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:08.579522 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:08.579555 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:08.579583 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:08.579699 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 00:37:08.579725 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:37:08.579979 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:08.576518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.31.210.254:42271)
0808 00:37:08.576686 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:08.576688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:08.576774 (+ 86us) server_negotiation.cc:407] Connection header received
0808 00:37:08.576842 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:08.576843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:08.576895 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:08.576974 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:08.577181 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.577980 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.578938 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:08.579242 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:08.579316 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:08.579528 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:08.579638 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:08.579701 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:08.579857 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:37:08.579889 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260808 00:37:08.581897 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 32237
I20260808 00:37:08.593377 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 32340
I20260808 00:37:08.600922 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 32438
2026-08-08T00:37:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3123 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:08 dist-test-slave-44d2 krb5kdc[32756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:08 dist-test-slave-44d2 krb5kdc[32756](info): set up 2 sockets
Aug 08 00:37:08 dist-test-slave-44d2 krb5kdc[32756](info): commencing operation
krb5kdc: starting...
W20260808 00:37:09.899652 32587 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/test-user.keytab.
Aug 08 00:37:09 dist-test-slave-44d2 krb5kdc[32756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.210.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
Entry for principal kudu/127.31.210.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab.
I20260808 00:37:10.035192 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39783
--webserver_interface=127.31.210.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:39783,127.31.210.253:45811,127.31.210.252:42567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:10.123008 309 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:37:10.123323 309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:10.123363 309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:10.123409 309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:10.124653 309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:10.124742 309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:10.124768 309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:10.124804 309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:10.124835 309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:10.126497 309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:39783,127.31.210.253:45811,127.31.210.252:42567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.254.keytab
--principal=kudu/127.31.210.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.254:39783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.210.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:10.127030 309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:10.127364 309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:10.132922 317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:10.132966 319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:10.132932 316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:10.133255 309 server_base.cc:1034] running on GCE node
I20260808 00:37:10.133483 309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:10.133785 309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:10.134948 309 hybrid_clock.cc:648] HybridClock initialized: now 1786149430134944 us; error 37 us; skew 500 ppm
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:10.137508 309 init.cc:377] Logged in from keytab as kudu/127.31.210.254@KRBTEST.COM (short username kudu)
I20260808 00:37:10.137931 309 webserver.cc:460] Webserver started at http://127.31.210.254:40255/ using document root <none> and password file <none>
I20260808 00:37:10.138180 309 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:10.138249 309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:10.138401 309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:10.139529 309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data/instance:
uuid: "2db4ded24c574874b076b539349a6886"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "abf07f15eb6bf690c4a334c8b4e6910f"
server_key_iv: "85cdde5170578ce25e309b24f81a93ae"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.139900 309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/wal/instance:
uuid: "2db4ded24c574874b076b539349a6886"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "abf07f15eb6bf690c4a334c8b4e6910f"
server_key_iv: "85cdde5170578ce25e309b24f81a93ae"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.141350 309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:10.142092 327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:10.142266 309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:10.142374 309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/wal
uuid: "2db4ded24c574874b076b539349a6886"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "abf07f15eb6bf690c4a334c8b4e6910f"
server_key_iv: "85cdde5170578ce25e309b24f81a93ae"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.142448 309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:10.171854 309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:10.173887 309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:10.174062 309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:10.178289 309 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.254:39783
I20260808 00:37:10.178350 396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.254:39783 every 8 connection(s)
I20260808 00:37:10.178728 309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:10.179641 397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:10.181452 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 309
I20260808 00:37:10.181319 397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:10.181569 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-0/wal/instance
I20260808 00:37:10.181814 32587 external_mini_cluster.cc:1428] Setting key 81da553fc141dcbaee891ee29eccbb25
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.210.253@KRBTEST.COM; defaulting to no policy
I20260808 00:37:10.191869 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.182068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39783 (local address 127.0.0.1:41926)
0808 00:37:10.182234 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.182276 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.182790 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.183126 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.183128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.183368 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.183523 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.183527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.184814 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.184816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.185680 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.185683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.186408 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.186409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.186467 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.187024 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.187032 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.188573 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.190382 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.190388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.190390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.190590 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.190848 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.190848 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.190849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.190944 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.191277 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.191283 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.191344 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.191454 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.191689 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":96,"thread_start_us":86,"threads_started":1}
I20260808 00:37:10.191871 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.182269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41926 (local address 127.31.210.254:39783)
0808 00:37:10.182439 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.182444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.182488 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:37:10.182961 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.182964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.182990 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.183106 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.183653 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.184718 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.185777 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.186295 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.186444 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.188671 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.188678 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.188678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.188685 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.190278 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.190686 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.190689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.190689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.190770 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.190996 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.190997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.190998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.191206 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.191298 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.191487 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.191623 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
I20260808 00:37:10.192648 397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:10.193023 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.192912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:46066)
0808 00:37:10.192959 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.193003 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:37:10.193179 333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
I20260808 00:37:10.193576 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.193491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:46074)
0808 00:37:10.193534 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.193560 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:37:10.193694 397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:45811: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab.
Entry for principal kudu/127.31.210.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab.
I20260808 00:37:10.212111 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:45811
--webserver_interface=127.31.210.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab
--principal=kudu/127.31.210.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:39783,127.31.210.253:45811,127.31.210.252:42567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:37:10.242980 397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } attempt: 1
I20260808 00:37:10.243630 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.243480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:44622)
0808 00:37:10.243552 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.243597 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 00:37:10.244059 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.243947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:44626)
0808 00:37:10.243997 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.244031 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:37:10.244166 397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:45811: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
W20260808 00:37:10.300278 407 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:37:10.300580 407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:10.300645 407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:10.300688 407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:10.301910 407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:10.301991 407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:10.302014 407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:10.302035 407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:10.302066 407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:10.303584 407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:39783,127.31.210.253:45811,127.31.210.252:42567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.253.keytab
--principal=kudu/127.31.210.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.253:45811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.210.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:10.303895 407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:10.304152 407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:10.309294 420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:10.309276 414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:10.309399 413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:10.309518 407 server_base.cc:1034] running on GCE node
I20260808 00:37:10.309731 407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:10.309973 407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:10.311129 407 hybrid_clock.cc:648] HybridClock initialized: now 1786149430311118 us; error 32 us; skew 500 ppm
I20260808 00:37:10.312342 397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } attempt: 2
I20260808 00:37:10.312903 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.312741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:44628)
0808 00:37:10.312811 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.312854 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 00:37:10.313392 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.313266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:44634)
0808 00:37:10.313324 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.313358 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:10.314414 407 init.cc:377] Logged in from keytab as kudu/127.31.210.253@KRBTEST.COM (short username kudu)
W20260808 00:37:10.314530 397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.253:45811: Network error: Client connection negotiation failed: client connection to 127.31.210.253:45811: connect: Connection refused (error 111)
I20260808 00:37:10.314869 407 webserver.cc:460] Webserver started at http://127.31.210.253:38893/ using document root <none> and password file <none>
I20260808 00:37:10.315099 407 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:10.315142 407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:10.315235 407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:10.316288 407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data/instance:
uuid: "ca88ccc4e81c481da63f517509c97486"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "b4d78f67b606bdf9cc4db5822180bb62"
server_key_iv: "96b28ef65d67f0865edbd990df6de704"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.316673 407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/wal/instance:
uuid: "ca88ccc4e81c481da63f517509c97486"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "b4d78f67b606bdf9cc4db5822180bb62"
server_key_iv: "96b28ef65d67f0865edbd990df6de704"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.318080 407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:10.318805 428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:10.319003 407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:10.319103 407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/wal
uuid: "ca88ccc4e81c481da63f517509c97486"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "b4d78f67b606bdf9cc4db5822180bb62"
server_key_iv: "96b28ef65d67f0865edbd990df6de704"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.319161 407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:10.338140 407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:10.340086 407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:10.340204 407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:10.344244 407 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.253:45811
I20260808 00:37:10.344301 498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.253:45811 every 8 connection(s)
I20260808 00:37:10.344731 407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/data/info.pb
I20260808 00:37:10.345739 505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:10.347363 505 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:10.348348 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 407
I20260808 00:37:10.348476 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-1/wal/instance
I20260808 00:37:10.348781 32587 external_mini_cluster.cc:1428] Setting key 9efda54d9c2c97d3e6679fa80baa9148
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.210.252@KRBTEST.COM; defaulting to no policy
I20260808 00:37:10.358156 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.348105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.31.210.254:39783)
0808 00:37:10.348198 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.348201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.348733 (+ 532us) server_negotiation.cc:407] Connection header received
0808 00:37:10.349485 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.349486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.349500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.349549 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.350402 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.351151 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.352302 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.352556 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.352643 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.355079 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.355082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.355083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.355094 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.356656 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.357093 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.357095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.357095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.357162 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.357448 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.357449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.357449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.357576 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.357654 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.357957 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.358058 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:37:10.358341 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.348068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39783 (local address 127.0.0.1:46574)
0808 00:37:10.348580 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.348632 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.349363 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.349631 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.349636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.350026 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.350314 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.350320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.351251 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.351252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.352218 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.352221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.352656 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.352657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.352807 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.353304 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.353315 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.354966 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.356753 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.356761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.356764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.356997 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.357250 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.357251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.357251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.357365 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.357644 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.357651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.357821 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.357934 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.358100 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":108,"threads_started":1}
I20260808 00:37:10.359077 505 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.210.252@KRBTEST.COM" created.
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:37:10.368067 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.359372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:44638)
0808 00:37:10.359425 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.359436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.359475 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.359791 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.359791 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.359931 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.359980 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.359981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.361302 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.361303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.362168 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.362172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.362678 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.362679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.362761 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.363314 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.363319 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.364858 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.366758 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.366759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.366760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.367010 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.367230 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.367231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.367231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.367308 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.367597 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.367598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.367663 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.367747 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.367857 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:37:10.368094 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.359413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.31.210.253:45811)
0808 00:37:10.359646 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.359650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.359668 (+ 18us) server_negotiation.cc:407] Connection header received
0808 00:37:10.359685 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.359686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.359698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.359777 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.360070 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.361070 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.362272 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.362595 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.362661 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.364948 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.364956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.364957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.364963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.366662 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.367105 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.367108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.367108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.367163 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.367375 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.367376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.367376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.367520 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.367584 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.367838 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.367914 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":156,"threads_started":1}
I20260808 00:37:10.368876 505 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:10.369297 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.369144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48656)
0808 00:37:10.369201 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.369260 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:37:10.369462 431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
I20260808 00:37:10.369947 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.369832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48668)
0808 00:37:10.369887 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.369919 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:37:10.370110 505 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:42567: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.210.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab.
Entry for principal kudu/127.31.210.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab.
I20260808 00:37:10.379935 32587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdt1wd3/build/release/bin/kudu
/tmp/dist-test-taskdt1wd3/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:42567
--webserver_interface=127.31.210.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab
--principal=kudu/127.31.210.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.210.212:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.210.254:39783,127.31.210.253:45811,127.31.210.252:42567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:37:10.405704 397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } attempt: 3
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:37:10.413017 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.406199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.31.210.253:45811)
0808 00:37:10.406270 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.406272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.406283 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:37:10.406334 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.406336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.406350 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.406427 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.406771 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.407279 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.408050 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.408280 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.408358 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.410365 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.410367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.410367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.410377 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.411787 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.412135 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.412137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.412137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.412198 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.412402 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.412403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.412403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.412585 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.412676 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.412837 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.412917 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:37:10.413028 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.406115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:44652)
0808 00:37:10.406189 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.406200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.406270 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.406433 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.406434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.406611 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.406682 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.406684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.407382 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.407388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.407918 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.407920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.408368 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.408369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.408459 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.409069 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.409074 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.410273 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.411875 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.411877 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.411877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.412058 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.412268 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.412269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.412270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.412336 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.412668 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.412669 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.412719 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.412811 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.412913 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:37:10.413441 397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:10.413861 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.413743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48682)
0808 00:37:10.413800 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.413834 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:37:10.414170 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.414084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48694)
0808 00:37:10.414129 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.414153 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:37:10.414297 397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:42567: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
I20260808 00:37:10.419355 505 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } attempt: 1
I20260808 00:37:10.419747 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.419615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48706)
0808 00:37:10.419672 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.419711 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:37:10.420087 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.419992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48714)
0808 00:37:10.420038 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.420066 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:37:10.420223 505 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:42567: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
I20260808 00:37:10.445413 397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } attempt: 1
I20260808 00:37:10.445942 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.445791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48728)
0808 00:37:10.445865 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.445909 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 00:37:10.446377 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.446223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48740)
0808 00:37:10.446289 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.446339 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 00:37:10.446517 397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:42567: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
W20260808 00:37:10.466526 516 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:37:10.466743 516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:10.466760 516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:10.466811 516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:10.467965 516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:10.468010 516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:10.468024 516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:10.468080 516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:10.468106 516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:10.469684 516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.210.212:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.210.254:39783,127.31.210.253:45811,127.31.210.252:42567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/krb5kdc/kudu_127.31.210.252.keytab
--principal=kudu/127.31.210.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.210.252:42567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.210.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:10.469959 516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:10.470227 516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:10.475262 526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:10.475262 532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:10.475327 528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:10.475448 516 server_base.cc:1034] running on GCE node
I20260808 00:37:10.475740 516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:10.475977 516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:10.477118 516 hybrid_clock.cc:648] HybridClock initialized: now 1786149430477102 us; error 32 us; skew 500 ppm
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:10.479867 516 init.cc:377] Logged in from keytab as kudu/127.31.210.252@KRBTEST.COM (short username kudu)
I20260808 00:37:10.480347 516 webserver.cc:460] Webserver started at http://127.31.210.252:39879/ using document root <none> and password file <none>
I20260808 00:37:10.480654 516 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:10.480707 516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:10.480809 516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:10.481928 516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data/instance:
uuid: "033f0705b0de4167b09353a2b0350ca0"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "3cd9513be59e89d4d3a04622692657af"
server_key_iv: "b807a4c04cbddb58c8f3a56f4aa11380"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.482268 516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/wal/instance:
uuid: "033f0705b0de4167b09353a2b0350ca0"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "3cd9513be59e89d4d3a04622692657af"
server_key_iv: "b807a4c04cbddb58c8f3a56f4aa11380"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.483767 516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:10.484666 544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:10.484849 516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:10.484965 516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data,/tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/wal
uuid: "033f0705b0de4167b09353a2b0350ca0"
format_stamp: "Formatted at 2026-08-08 00:37:10 on dist-test-slave-44d2"
server_key: "3cd9513be59e89d4d3a04622692657af"
server_key_iv: "b807a4c04cbddb58c8f3a56f4aa11380"
server_key_version: "encryptionkey@0"
I20260808 00:37:10.485066 516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:10.488394 505 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } attempt: 2
I20260808 00:37:10.488845 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.488714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48748)
0808 00:37:10.488776 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.488816 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:37:10.489252 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.489155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48750)
0808 00:37:10.489210 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.489236 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 00:37:10.489362 505 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.210.252:42567: Network error: Client connection negotiation failed: client connection to 127.31.210.252:42567: connect: Connection refused (error 111)
I20260808 00:37:10.502211 516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:10.504258 516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:10.504428 516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:10.508366 516 rpc_server.cc:307] RPC server started. Bound to: 127.31.210.252:42567
I20260808 00:37:10.508402 613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.210.252:42567 every 8 connection(s)
I20260808 00:37:10.508854 516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/data/info.pb
I20260808 00:37:10.509756 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:10.511296 614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:10.515760 32587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdt1wd3/build/release/bin/kudu as pid 516
I20260808 00:37:10.515883 32587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdt1wd3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149138237870-32587-0/minicluster-data/master-2/wal/instance
I20260808 00:37:10.516124 32587 external_mini_cluster.cc:1428] Setting key 16f37b11cfb4a3fef98a6c08430c7d85
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:37:10.520947 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.512034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46576 (local address 127.31.210.254:39783)
0808 00:37:10.512082 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.512084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.512318 (+ 234us) server_negotiation.cc:407] Connection header received
0808 00:37:10.512913 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.512914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.512926 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.512980 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.513616 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.514093 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.515088 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.515292 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.515396 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.517843 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.517844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.517845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.517853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.519265 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.519785 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.519786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.519786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.519856 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.520107 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.520108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.520108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.520263 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.520335 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.520761 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.520861 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:10.521185 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.511992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.254:39783 (local address 127.0.0.1:46576)
0808 00:37:10.512192 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.512230 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.512799 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.513058 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.513062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.513319 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.513515 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.513519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.514190 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.514191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.514973 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.514975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.515404 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.515405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.515542 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.516108 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.516116 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.517759 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.519351 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.519359 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.519362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.519707 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.519928 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.519929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.519929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.520039 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.520338 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.520347 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.520611 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.520706 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.520929 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260808 00:37:10.521651 397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } attempt: 2
I20260808 00:37:10.521914 614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
I20260808 00:37:10.526952 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.516543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.31.210.252:42567)
0808 00:37:10.516712 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.516716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.516793 (+ 77us) server_negotiation.cc:407] Connection header received
0808 00:37:10.516856 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.516858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.516881 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.516970 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.517653 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.518729 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.519698 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.520178 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.520253 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.523472 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.523480 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.523481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.523488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.525573 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.526016 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.526020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.526021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.526081 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.526280 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.526282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.526282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.526497 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.526685 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.526736 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.526819 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.254@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:37:10.531675 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.522253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44658 (local address 127.31.210.253:45811)
0808 00:37:10.522301 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.522303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.522360 (+ 57us) server_negotiation.cc:407] Connection header received
0808 00:37:10.522468 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.522469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.522479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.522529 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.522939 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.523638 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.524541 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.524865 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.524945 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.528320 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.528322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.528322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.528331 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.530243 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.530711 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.530712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.530713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.530776 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.531045 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.531047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.531047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.531197 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.531236 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.531463 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.531561 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:10.531998 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.522179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.253:45811 (local address 127.0.0.1:44658)
0808 00:37:10.522303 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.522314 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.522386 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.522594 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.522594 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.522767 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.522856 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.522858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.523699 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.523700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.524433 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.524436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.524988 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.524990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.525076 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.525649 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.525655 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.528220 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.530330 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.530331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.530332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.530622 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.530854 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.530855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.530856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.530941 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.531279 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.531280 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.531316 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.531767 (+ 451us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.531881 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 00:37:10.532382 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.522408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.31.210.252:42567)
0808 00:37:10.522564 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.522566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.522575 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:10.522597 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.522598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.522609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.522692 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.523013 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.523545 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.524560 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.525149 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.525237 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.527913 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.527914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.527915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.527922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.531017 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.531388 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.531389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.531390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.531468 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.531686 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.531688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.531688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.531828 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.531893 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.532200 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.532287 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260808 00:37:10.532387 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.522314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48766)
0808 00:37:10.522356 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.522367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.522405 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.522708 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.522709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.522859 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.522919 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.522921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.523632 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.523633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.524426 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.524430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.525262 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.525263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.525327 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.525890 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.525896 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.527811 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.531091 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.531092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.531092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.531307 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.531553 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.531555 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.531555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.531628 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.532019 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.532020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.532077 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.532152 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.532250 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 00:37:10.532701 614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.254@KRBTEST.COM
I20260808 00:37:10.534893 397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886: Bootstrap starting.
I20260808 00:37:10.536211 397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:10.536618 397 log.cc:824] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:10.537009 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.527801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46590 (local address 127.31.210.254:39783)
0808 00:37:10.527843 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.527845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.527853 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:10.527875 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.527876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.527888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.527954 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.528619 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.529431 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.531034 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.531321 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.531382 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.534043 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.534044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.534045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.534052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.535745 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.536091 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.536093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.536093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.536159 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.536400 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.536401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.536401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.536599 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.536769 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.536816 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.536888 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:10.537768 397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886: No bootstrap required, opened a new log
I20260808 00:37:10.539404 397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.539534 397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db4ded24c574874b076b539349a6886, State: Initialized, Role: FOLLOWER
I20260808 00:37:10.539629 397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.539942 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.540031 397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:10.540045 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.210.253@KRBTEST.COM
W20260808 00:37:10.542039 637 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:10.542693 637 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.252@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:37:10.546247 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.537622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44668 (local address 127.31.210.253:45811)
0808 00:37:10.537669 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.537672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.537692 (+ 20us) server_negotiation.cc:407] Connection header received
0808 00:37:10.537712 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.537713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.537724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.537787 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.538274 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.538736 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.539406 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.539660 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.539733 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.542325 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.542326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.542327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.542334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.544343 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.544822 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.544824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.544824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.544881 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.545070 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.545071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.545071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.545197 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.545359 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.545801 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.546150 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:37:10.543476 309 master_runner.cc:186] Error getting master registration for 127.31.210.253:45811: OK, instance_id { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" instance_seqno: 1786149430343129 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:37:10.546818 309 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:37:10.548038 32587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:10.548606 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.532984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48772)
0808 00:37:10.533017 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.533028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.533072 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.533206 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.533207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.533349 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.533375 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.533376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.542083 (+ 8707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.542084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.542903 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.542907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.543497 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.543498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.543552 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.543949 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.543954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.545523 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.547448 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.547449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.547449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.547679 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.547927 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.547928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.547928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.547999 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.548305 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.548306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.548337 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.548377 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.548460 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 00:37:10.548913 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.533020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.31.210.252:42567)
0808 00:37:10.533054 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.533056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.533065 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:10.533134 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.533135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.533145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.533203 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.536903 (+ 3700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.541962 (+ 5059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.543086 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.543404 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.543704 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.545621 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.545623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.545623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.545630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.547357 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.547766 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.547767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.547767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.547833 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.548070 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.548072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.548072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.548237 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.548714 (+ 477us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.548752 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.548830 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:10.549119 368 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2db4ded24c574874b076b539349a6886"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46590
I20260808 00:37:10.549257 368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 00:37:10.549310 368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:10.550014 368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.550285 368 leader_election.cc:288] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [CANDIDATE]: Term 1 election: Requested vote from peers ca88ccc4e81c481da63f517509c97486 (127.31.210.253:45811), 033f0705b0de4167b09353a2b0350ca0 (127.31.210.252:42567)
I20260808 00:37:10.550535 614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: Bootstrap starting.
I20260808 00:37:10.551388 614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:10.551744 614 log.cc:824] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:10.552790 614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: No bootstrap required, opened a new log
I20260808 00:37:10.555433 614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.555616 614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 033f0705b0de4167b09353a2b0350ca0, State: Initialized, Role: FOLLOWER
I20260808 00:37:10.555716 614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.556067 614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:10.556142 575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db4ded24c574874b076b539349a6886" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "033f0705b0de4167b09353a2b0350ca0"
I20260808 00:37:10.556267 575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:10.557262 575 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2db4ded24c574874b076b539349a6886 in term 1.
I20260808 00:37:10.557614 330 leader_election.cc:302] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 033f0705b0de4167b09353a2b0350ca0, 2db4ded24c574874b076b539349a6886; no voters:
I20260808 00:37:10.557837 624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:10.558034 624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [term 1 LEADER]: Becoming Leader. State: Replica: 2db4ded24c574874b076b539349a6886, State: Running, Role: LEADER
I20260808 00:37:10.558154 624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.558684 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db4ded24c574874b076b539349a6886. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.558784 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: This master's current role is: LEADER
W20260808 00:37:10.558328 669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:10.559257 669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:10.556077 649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.561069 649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:10.561630 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.553140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.31.210.254:39783)
0808 00:37:10.553184 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.553186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.553545 (+ 359us) server_negotiation.cc:407] Connection header received
0808 00:37:10.553660 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.553661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.553688 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.553720 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.554508 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.555041 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.556204 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.556551 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.556599 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.558377 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.558379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.558379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.558388 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.560251 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.560698 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.560699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.560700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.560761 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.560988 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.560989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.560989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.561135 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.561221 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.561433 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.561522 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:10.561625 516 master_runner.cc:186] Error getting master registration for 127.31.210.253:45811: OK, instance_id { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" instance_seqno: 1786149430343129 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:37:10.564589 516 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:37:10.567399 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.553229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.31.210.253:45811)
0808 00:37:10.553270 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.553272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.558748 (+ 5476us) server_negotiation.cc:407] Connection header received
0808 00:37:10.558947 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.558949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.558964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.559000 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.559908 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.560786 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.561925 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.562252 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.562299 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.564028 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.564030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.564030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.564038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.565972 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.566420 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.566422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.566422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.566485 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.566721 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.566722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.566723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.566879 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.567028 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.567190 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.567271 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:37:10.569274 675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:10.569520 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.559736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.31.210.252:42567)
0808 00:37:10.559784 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.559787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.561716 (+ 1929us) server_negotiation.cc:407] Connection header received
0808 00:37:10.561819 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.561821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.561831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.561861 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.562540 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.563149 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.564111 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.564373 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.564437 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.566557 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.566558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.566559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.566566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.568280 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.568684 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.568685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.568686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.568739 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.568983 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.568984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.568984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.569097 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.569205 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.569320 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.569403 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:37:10.569546 675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:10.571470 575 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [term 1 FOLLOWER]: Refusing update from remote peer 2db4ded24c574874b076b539349a6886: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:10.571813 670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:10.573218 649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db4ded24c574874b076b539349a6886. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.573320 649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:10.574404 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.574491 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:10.574640 649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.574715 649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:10.575151 681 mvcc.cc:205] Tried to move back new op lower bound from 7316068067618328576 to 7316068067567259648. Current Snapshot: MvccSnapshot[applied={T|T < 7316068067618328576}]
I20260808 00:37:10.576027 675 catalog_manager.cc:1347] Generated new cluster ID: cdefb7cc03634d96a0c0d3f368c54c4e
I20260808 00:37:10.576082 675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:10.580508 505 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } attempt: 3
Aug 08 00:37:10 dist-test-slave-44d2 krb5kdc[32756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.210.253@KRBTEST.COM for kudu/127.31.210.252@KRBTEST.COM
I20260808 00:37:10.589380 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.580992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.31.210.252:42567)
0808 00:37:10.581062 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.581064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.581072 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:10.581092 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.581093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.581102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.581156 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.581542 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.582263 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.583392 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.583779 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.583854 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.586321 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:10.586322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:10.586323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:10.586330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:10.588137 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.588583 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.588584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.588585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.588645 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:10.588842 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:10.588843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:10.588843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:10.588972 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:10.589036 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.589216 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.589285 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:37:10.589380 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.580735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.210.252:42567 (local address 127.0.0.1:48788)
0808 00:37:10.580798 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:10.580813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:10.580893 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:10.581172 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:10.581173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:10.581405 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:10.581468 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.581470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.582333 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.582335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.583283 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:10.583286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.583864 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.583865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:10.583927 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.584546 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:10.584551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:10.586230 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:10.588228 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.588229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.588230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.588449 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.588705 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:10.588706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:10.588706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:10.588773 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:10.589062 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:10.589063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:10.589109 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 00:37:10.589178 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:37:10.589283 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:37:10.589627 675 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:10.591342 505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486: Bootstrap starting.
I20260808 00:37:10.592101 505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:10.592424 505 log.cc:824] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:10.593469 675 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:10.599889 505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486: No bootstrap required, opened a new log
I20260808 00:37:10.602840 505 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.603004 505 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca88ccc4e81c481da63f517509c97486, State: Initialized, Role: FOLLOWER
I20260808 00:37:10.603096 505 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } }
I20260808 00:37:10.603910 505 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:10.603948 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db4ded24c574874b076b539349a6886" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ca88ccc4e81c481da63f517509c97486"
I20260808 00:37:10.604054 463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:10.605116 463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2db4ded24c574874b076b539349a6886 in term 1.
I20260808 00:37:10.605463 462 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [term 1 FOLLOWER]: Refusing update from remote peer 2db4ded24c574874b076b539349a6886: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:10.605850 670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:10.608179 706 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.608417 706 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:10.608525 706 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db4ded24c574874b076b539349a6886. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.608577 706 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:10.608604 706 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:10.608634 706 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:10.608923 675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886: Generated new TSK 0
I20260808 00:37:10.609263 675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:10.610839 701 catalog_manager.cc:1259] Loaded cluster ID: cdefb7cc03634d96a0c0d3f368c54c4e
I20260808 00:37:10.610890 701 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486: loading cluster ID for follower catalog manager: success
I20260808 00:37:10.612038 701 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486: acquiring CA information for follower catalog manager: success
I20260808 00:37:10.612360 701 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca88ccc4e81c481da63f517509c97486: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 00:37:10.639796 511 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.637704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.31.210.253:45811)
0808 00:37:10.637760 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.637763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.638928 (+ 1165us) server_negotiation.cc:407] Connection header received
0808 00:37:10.639072 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.639074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.639088 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.639124 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.639704 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44696: BlockingRecv error: recv got EOF from 127.0.0.1:44696 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 00:37:10.640817 400 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.638183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46618 (local address 127.31.210.254:39783)
0808 00:37:10.638233 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.638235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.640134 (+ 1899us) server_negotiation.cc:407] Connection header received
0808 00:37:10.640250 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.640252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.640266 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.640300 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.640738 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46618: BlockingRecv error: recv got EOF from 127.0.0.1:46618 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 00:37:10.641772 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.638576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.31.210.252:42567)
0808 00:37:10.638622 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.638624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.641082 (+ 2458us) server_negotiation.cc:407] Connection header received
0808 00:37:10.641196 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.641197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.641211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.641245 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.641693 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48792: BlockingRecv error: recv got EOF from 127.0.0.1:48792 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:37:10.655064 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.645377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.31.210.253:45811)
0808 00:37:10.645421 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.645423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.649050 (+ 3627us) server_negotiation.cc:407] Connection header received
0808 00:37:10.649440 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.649442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.649534 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.649628 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.650460 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.651631 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.653951 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.654281 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.654353 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.654611 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.654765 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.654823 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.654951 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.654974 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 00:37:10.658973 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.658065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.31.210.252:42567)
0808 00:37:10.658114 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.658116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.658330 (+ 214us) server_negotiation.cc:407] Connection header received
0808 00:37:10.658469 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.658470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.658493 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.658527 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.658912 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48806: BlockingRecv error: recv got EOF from 127.0.0.1:48806 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:37:10.665134 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.661500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.31.210.253:45811)
0808 00:37:10.661540 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.661542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.661830 (+ 288us) server_negotiation.cc:407] Connection header received
0808 00:37:10.661936 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.661937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.661948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.661979 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.662159 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.662990 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.664209 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.664586 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.664632 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.664848 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.664883 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.664913 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.665029 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.665051 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 00:37:10.668602 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.667706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.31.210.252:42567)
0808 00:37:10.667747 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.667749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.667960 (+ 211us) server_negotiation.cc:407] Connection header received
0808 00:37:10.668062 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.668063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.668074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.668105 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.668545 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48816: BlockingRecv error: recv got EOF from 127.0.0.1:48816 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:10.675484 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.671880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.31.210.253:45811)
0808 00:37:10.671922 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.671925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.672201 (+ 276us) server_negotiation.cc:407] Connection header received
0808 00:37:10.672340 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.672342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.672361 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.672394 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.672677 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.673507 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.674623 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.674911 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.674959 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.675179 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.675216 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.675246 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.675367 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.675386 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 00:37:10.679297 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.678344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.31.210.252:42567)
0808 00:37:10.678394 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.678396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.678649 (+ 253us) server_negotiation.cc:407] Connection header received
0808 00:37:10.678768 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.678769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.678783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.678819 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.679231 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48830: BlockingRecv error: recv got EOF from 127.0.0.1:48830 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:10.688777 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.684735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44732 (local address 127.31.210.253:45811)
0808 00:37:10.684785 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.684787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.685082 (+ 295us) server_negotiation.cc:407] Connection header received
0808 00:37:10.685201 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.685202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.685215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.685249 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.685455 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.686242 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.687624 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.687961 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.688014 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.688344 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.688382 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.688425 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.688664 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.688688 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 00:37:10.692358 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.691440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.31.210.252:42567)
0808 00:37:10.691486 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.691488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.691718 (+ 230us) server_negotiation.cc:407] Connection header received
0808 00:37:10.691833 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.691834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.691848 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.691882 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.692299 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48836: BlockingRecv error: recv got EOF from 127.0.0.1:48836 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:10.706151 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.701956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44736 (local address 127.31.210.253:45811)
0808 00:37:10.702006 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.702009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.702334 (+ 325us) server_negotiation.cc:407] Connection header received
0808 00:37:10.702476 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.702478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.702491 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.702525 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.702743 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.703636 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.705037 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.705363 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.705437 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.705840 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.705881 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.705914 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.706041 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.706065 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:37:10.709825 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.708896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48840 (local address 127.31.210.252:42567)
0808 00:37:10.708944 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.708946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.709176 (+ 230us) server_negotiation.cc:407] Connection header received
0808 00:37:10.709290 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.709292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.709305 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.709339 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.709767 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48840: BlockingRecv error: recv got EOF from 127.0.0.1:48840 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:10.731087 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.727342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.31.210.253:45811)
0808 00:37:10.727398 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.727401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.727782 (+ 381us) server_negotiation.cc:407] Connection header received
0808 00:37:10.727896 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.727897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.727909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.727938 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.728171 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.728922 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.730053 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.730494 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.730546 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.730805 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.730842 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.730870 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.730979 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.731001 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 00:37:10.734686 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.733761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.31.210.252:42567)
0808 00:37:10.733804 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.733806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.734074 (+ 268us) server_negotiation.cc:407] Connection header received
0808 00:37:10.734198 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.734199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.734212 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.734244 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.734630 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48852: BlockingRecv error: recv got EOF from 127.0.0.1:48852 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:10.770695 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.768121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.31.210.253:45811)
0808 00:37:10.768176 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.768178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.768343 (+ 165us) server_negotiation.cc:407] Connection header received
0808 00:37:10.768426 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.768428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.768442 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.768527 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.768678 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.769161 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.769962 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.770159 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.770235 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.770399 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.770439 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.770500 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.770560 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.770581 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:37:10.773504 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.772802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.31.210.252:42567)
0808 00:37:10.772855 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.772857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.772976 (+ 119us) server_negotiation.cc:407] Connection header received
0808 00:37:10.773055 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.773056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.773067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.773111 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.773449 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48854: BlockingRecv error: recv got EOF from 127.0.0.1:48854 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:37:10.841570 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.838647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.31.210.253:45811)
0808 00:37:10.838733 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.838735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.838837 (+ 102us) server_negotiation.cc:407] Connection header received
0808 00:37:10.838944 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.838946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.838958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.839030 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.839185 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.839684 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.840585 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.840780 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.840862 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.841275 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.841319 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.841385 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.841443 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.841465 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 00:37:10.844204 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.843517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.31.210.252:42567)
0808 00:37:10.843582 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.843585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.843640 (+ 55us) server_negotiation.cc:407] Connection header received
0808 00:37:10.843682 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.843683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.843693 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.843738 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.844119 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48870: BlockingRecv error: recv got EOF from 127.0.0.1:48870 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:37:10.975965 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:10.973455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44764 (local address 127.31.210.253:45811)
0808 00:37:10.973569 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.973571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.973620 (+ 49us) server_negotiation.cc:407] Connection header received
0808 00:37:10.973676 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.973677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.973687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.973745 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:10.973876 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.974427 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.975220 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:10.975453 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:10.975514 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:10.975663 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:10.975694 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:10.975737 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:10.975812 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:37:10.975830 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 00:37:10.978540 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:10.977815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.31.210.252:42567)
0808 00:37:10.977897 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:10.977900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:10.977944 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:37:10.978043 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:10.978044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:10.978056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:10.978119 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:10.978475 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48884: BlockingRecv error: recv got EOF from 127.0.0.1:48884 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:37:11.011622 670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2db4ded24c574874b076b539349a6886" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db4ded24c574874b076b539349a6886" member_type: VOTER last_known_addr { host: "127.31.210.254" port: 39783 } } peers { permanent_uuid: "ca88ccc4e81c481da63f517509c97486" member_type: VOTER last_known_addr { host: "127.31.210.253" port: 45811 } } peers { permanent_uuid: "033f0705b0de4167b09353a2b0350ca0" member_type: VOTER last_known_addr { host: "127.31.210.252" port: 42567 } } }
I20260808 00:37:11.011804 670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db4ded24c574874b076b539349a6886 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:11.238448 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:11.235829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.31.210.253:45811)
0808 00:37:11.236012 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:11.236014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:11.236022 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:11.236060 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:11.236061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:11.236070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:11.236146 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:11.236317 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.236866 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.237693 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.237902 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.237995 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:11.238166 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:11.238205 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:11.238269 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:11.238334 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 00:37:11.238356 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
W20260808 00:37:11.241312 621 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:11.240429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48890 (local address 127.31.210.252:42567)
0808 00:37:11.240538 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:11.240540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:11.240698 (+ 158us) server_negotiation.cc:407] Connection header received
0808 00:37:11.240798 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:11.240799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:11.240814 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:11.240873 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:11.241246 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48890: BlockingRecv error: recv got EOF from 127.0.0.1:48890 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:37:11.559798 669 catalog_manager.cc:1259] Loaded cluster ID: cdefb7cc03634d96a0c0d3f368c54c4e
I20260808 00:37:11.559911 669 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: loading cluster ID for follower catalog manager: success
I20260808 00:37:11.560851 669 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: acquiring CA information for follower catalog manager: success
I20260808 00:37:11.561175 669 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 033f0705b0de4167b09353a2b0350ca0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:37:11.757236 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:11.754567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.31.210.253:45811)
0808 00:37:11.754685 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:11.754687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:11.754715 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:37:11.754800 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:11.754801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:11.754812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:11.754872 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:11.755009 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.755595 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.756340 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.756655 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.756712 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:11.756912 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:11.756951 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:11.757012 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:11.757087 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:37:11.757110 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":63,"threads_started":1}
I20260808 00:37:11.761735 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:11.759225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.31.210.252:42567)
0808 00:37:11.759336 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:11.759338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:11.759394 (+ 56us) server_negotiation.cc:407] Connection header received
0808 00:37:11.759452 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:11.759453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:11.759504 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:11.759578 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:11.759749 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.760317 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.761076 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.761278 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.761332 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:11.761457 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:11.761565 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:11.761619 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:11.761656 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 00:37:11.761671 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":64,"threads_started":1}
I20260808 00:37:11.766345 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:11.763656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.31.210.252:42567)
0808 00:37:11.763698 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:11.763700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:11.763740 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:37:11.763837 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:11.763838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:11.763847 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:11.763891 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:11.763986 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.764739 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.765611 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.765819 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.765902 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:11.766086 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:11.766119 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:11.766144 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:11.766242 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 00:37:11.766264 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:37:11.767007 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:11.763530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.31.210.254:39783)
0808 00:37:11.763758 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:11.763760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:11.763767 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:11.763807 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:11.763808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:11.763846 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:11.763946 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:11.764117 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.764895 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.765974 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.766230 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.766309 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:11.766663 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:11.766781 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:11.766852 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:11.766909 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:37:11.766940 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260808 00:37:11.767186 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:11.763525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.31.210.253:45811)
0808 00:37:11.763707 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:11.763709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:11.764095 (+ 386us) server_negotiation.cc:407] Connection header received
0808 00:37:11.764221 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:11.764223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:11.764234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:11.764289 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:11.764407 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.765048 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.766028 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:11.766315 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:11.766381 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:11.766756 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:11.766786 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:11.766845 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:11.767068 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 00:37:11.767095 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260808 00:37:11.768958 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 309
I20260808 00:37:11.781961 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 407
I20260808 00:37:11.794498 32587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdt1wd3/build/release/bin/kudu with pid 516
2026-08-08T00:37:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3191 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6315 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293560 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST